Main
Language
Runtime
Tools
FAQs
Changes
Download
Licence
cheesy

The Idle Language

This part of the Idle documentation describes the lexis, syntax, and semantics of Idle. In other words, this page describes which tokens are valid, how they can be combined, and what their combinations mean. The language constructs will be explained using the usual extended BNF notation, in which {a} means 0 or more a's, and [a] means an optional a. The Syntax of Idle can be found at the end of this manual.

As Idle is based on Lua, anyone familiar with Lua will already know the basics of Idle. See Differences to Lua for the most important changes and additions as well as a few hints on re-using existing Lua source code within Idle.

Another big chunk of documentation deals with the various parts of the main Idle Runtime library.

Note: if you find errors, omissions, obscurity or even simple typos... please send a short bug report to idle.script@gmail.com or idle.script@idle.thomaslauer.com. Getting the documentation up to scratch is high on my list of priorities!

Lexical Conventions
top of page

Names (also called identifiers) in Idle can be any string of letters, digits, and underscores, not beginning with a digit. This coincides with the definition of names in most languages. (The definition of letter depends on the current locale: any character considered alphabetic by the current locale can be used in an identifier.) Identifiers are used to name variables and table fields.

The following keywords are reserved and cannot be used as names (there is a token filter extension that allows keywords to be used as names for table fields):

and   break     do      else   elseif  end    false
for   function  if      in     local   nil    not
or    repeat    return  then   true    until  while

Idle is a case-sensitive language: elseif is a reserved word, but Elseif and ELSEIF are two different, valid names. As a convention, names starting with an underscore followed by uppercase letters (such as _VERSION or _PROMPT) are reserved for internal global variables used by Idle.

The following strings denote other tokens:

+     -     *     /     %     ^     #     ==    ~=
!=    <=    >=    <     >     =     (     )     {
}     [     ]     ;     :     ,     .     ..    ...

Literal strings can be delimited by matching single or double quotes, and can contain the following C-like escape sequences: '\a' (bell), '\b' (backspace), '\f' (form feed), '\n' (newline), '\r' (carriage return), '\t' (horizontal tab), '\v' (vertical tab), '\\' (backslash), '\"' (quotation mark [double quote]), and '\'' (apostrophe [single quote]). Moreover, a backslash followed by a real newline results in a newline in the string. A character in a string may also be specified by its numerical value, using either the escape sequence \ddd, where ddd is a sequence of up to three decimal digits (note that if such a numerical escape is to be followed by a digit, it must be expressed using exactly three decimal digits) or the escape sequence \xXX, where XX is a sequence of exactly two hexadecimal digits. Strings in Idle may contain any 8-bit value, including embedded zeros, which can be specified as '\0' or '\x00'.

To put a double (single) quote, a newline, a backslash, or an embedded zero inside a literal string enclosed by double (single) quotes you must use an escape sequence. Any other character may be directly inserted into the literal. (Some control characters may cause problems for the file system, but Idle has no problem with them.)

Literal strings can also be defined using a long format enclosed by either backticks (`...`) or long brackets. We define an opening long bracket of level n as an opening square bracket followed by n equal signs followed by another opening square bracket. So, an opening long bracket of level 0 is written as [[, an opening long bracket of level 1 is written as [=[, and so on. A closing long bracket is defined similarly; for instance, a closing long bracket of level 4 is written as ]====]. A long string starts with an opening long bracket of any level and ends at the first closing long bracket of the same level. Literals in this bracketed form may run for several lines, do not interpret any escape sequences, and ignore long brackets of any other level. They may contain anything except a closing bracket of the proper level.

For convenience, when the opening long bracket is immediately followed by a newline, the newline is not included in the string. As an example, in a system using ASCII (in which 'a' is coded as 97, newline is coded as 10, and '1' is coded as 49), the five literals below denote the same string:

a='alo\n123"'
a="alo\n123\""
a='\97lo\10\04923"'
a=[=[alo
123"]=]
a=`alo
123"`

A numerical constant may be written with an optional decimal part and an optional decimal exponent. Idle also accepts integer hexadecimal constants, by prefixing them with 0x. Examples of valid numerical constants are

3   3.0   3.1416   314.16e-2   0.31416E1   0xff   0x56

A comment starts with a double hyphen (--) anywhere outside a string. If the text immediately after -- is not an opening long bracket, the comment is a short comment, which runs until the end of the line. Otherwise, it is a long comment, which runs until the corresponding closing long bracket. Long comments are frequently used to disable code temporarily:

--[=[
-- nested short comment
x=4
print(x)
--]=]

Values and Types
top of page

Idle is a dynamically typed language. This means that variables do not have types; only values do. There are no type definitions in the language. All values carry their own type. All values in Idle are first-class values. This means that all values can be stored in variables, passed as arguments to other functions, and returned as results.

There are eight basic types in Idle: nil, boolean, number, string, function, userdata, thread, and table. Nil is the type of the value nil, whose main property is to be different from any other value; it usually represents the absence of a useful value. Boolean is the type of the values false and true. Both nil and false make a condition false; any other value makes it true. Number represents real (double-precision floating-point) numbers. String represents arrays of characters. Idle is 8-bit clean: strings may contain any 8-bit character, including embedded zeros ('\0').

Idle can call (and manipulate) functions written in Idle and functions written in C (see Function Calls).

The type userdata is provided to allow arbitrary C data to be stored in Idle variables. This type corresponds to a block of raw memory and has no pre-defined operations in Idle, except assignment and identity test. However, by using metatables, the programmer can define operations for userdata values (see Metatables). Userdata values cannot be created or modified in Idle, only through the C API. This guarantees the integrity of data owned by the host program.

The type thread represents independent threads of execution and it is used to implement coroutines (see Coroutines). Do not confuse Idle threads with operating-system threads (see module task for the latter). Idle supports coroutines on all systems, even those that do not support threads.

The type table implements associative arrays, that is, arrays that can be indexed not only with numbers, but with any value (except nil). Tables can be heterogeneous; that is, they can contain values of all types (except nil). Tables are the sole data structuring mechanism in Idle; they may be used to represent ordinary arrays, symbol tables, sets, records, graphs, trees, et cetera. To represent records, Idle uses the field name as an index. The language supports this representation by providing a.name as syntactic sugar for a["name"]. There are several convenient ways to create tables in Idle (see Table Constructors).

Like indices, the value of a table field can be of any type (except nil). In particular, because functions are first-class values, table fields may contain functions. Thus tables may also carry methods (see Function Definitions).

Tables, functions, threads, and (full) userdata values are objects: variables do not actually contain these values, only references to them. Assignment, parameter passing, and function returns always manipulate references to such values; these operations do not imply any kind of copy.

The core library function type( ) returns a string describing the type of a given value.

Coercion
top of page

Idle provides automatic conversion between string and number values at run time. Any arithmetic operation applied to a string tries to convert this string to a number, following the usual conversion rules. Conversely, whenever a number is used where a string is expected, the number is converted to a string, in a reasonable format. For complete control over how numbers are converted to strings, use the format( ) function from the string library (see string.format( )).

Variables
top of page

Variables are places that store values. There are three kinds of variables in Idle: global variables, local variables, and table fields.

A single name can denote a global variable or a local variable (or a function's formal parameter, which is a particular kind of local variable):

var::=Name

Name denotes identifiers, as defined in Lexical Conventions.

Any variable is assumed to be global unless explicitly declared as a local (see Local Declarations). Local variables are lexically scoped: local variables can be freely accessed by functions defined inside their scope (see Visibility Rules). Before the first assignment to a variable, its value is nil.

Square brackets are used to index a table:

var::=prefixexp'['exp']'

The meaning of accesses to global variables and table fields can be changed via metatables. An access to an indexed variable t[i] is equivalent to a call gettable_event(t,i). (See Metatables for a complete description of the gettable_event function. This function is not defined or callable in Idle. We use it here only for explanatory purposes.)

The syntax var.Name is syntactic sugar for var["Name"]:

var::=prefixexp'.'Name

All global variables live as fields in ordinary Idle tables, called environment tables or simply environments (see Environments). Each function has its own reference to an environment, so that all global variables in this function will refer to this environment table. When a function is created, it inherits the environment from the function that created it. To get the environment table of an Idle function, you call getfenv. To replace it, you call setfenv. (You can only manipulate the environment of C functions through the debug library.)

An access to a global variable x is equivalent to _env.x, which in turn is equivalent to

gettable_event(_env,"x")

where _env is the environment of the running function. (See Metatables for a complete description of the gettable_event function. This function is not defined or callable in Idle. Similarly, the _env variable is not defined in Idle. We use them here only for explanatory purposes.)

Statements
top of page

Idle supports an almost conventional set of statements, similar to those in Pascal or C. This set includes assignment, control structures, function calls, and variable declarations.

Chunks
top of page

The unit of execution of Idle is called a chunk. A chunk is simply a sequence of statements, which are executed sequentially. Each statement can be optionally followed by a semicolon:

chunk::={stat[';']}

There are no empty statements; thus ';;' is not legal.

Idle handles a chunk as the body of an anonymous function with a variable number of arguments (see Function Definitions). As such, chunks can define local variables, receive arguments, and return values.

A chunk may be stored in a file or in a string inside the host program. When a chunk is executed, first it is pre-compiled into instructions for a virtual machine, and then the compiled code is executed by an interpreter for the virtual machine.

Chunks may also be pre-compiled into binary form; see program idlec for details. Programs in source and compiled forms are interchangeable; Idle automatically detects the file type and acts accordingly.

Blocks
top of page

A block is a list of statements; syntactically, a block is the same as a chunk:

block::=chunk'

A block may be explicitly delimited to produce a single statement:

stat::=do block end

Explicit blocks are useful to control the scope of variable declarations. Explicit blocks are also sometimes used to add a return or break statement in the middle of another block (see Control Structures).

Assignment
top of page

Idle allows multiple assignment. Therefore, the syntax for assignment defines a list of variables on the left side and a list of expressions on the right side. The elements in both lists are separated by commas:

stat::=varlist1'='explist1
varlist1::=var{','var}
explist1::=exp{','exp}

Expressions are discussed in Expressions.

Before the assignment, the list of values is adjusted to the length of the list of variables. If there are more values than needed, the excess values are thrown away. If there are fewer values than needed, the list is extended with as many nil's as needed. If the list of expressions ends with a function call, then all values returned by this call enter in the list of values, before the adjustment (except when the call is enclosed in parentheses; see Expressions).

The assignment statement first evaluates all its expressions and only then are the assignments performed. Thus the code

i=3
i,a[i]=i+1,20

sets a[3] to 20, without affecting a[4] because the i in a[i] is evaluated (to 3) before it is assigned 4. Similarly, the line

x,y=y,x

exchanges the values of x and y.

The meaning of assignments to global variables and table fields can be changed via metatables. An assignment to an indexed variable t[i] = val is equivalent to settable_event(t,i,val). (See Metatables for a complete description of the settable_event function. This function is not defined or callable in Idle. We use it here only for explanatory purposes.)

An assignment to a global variable x = val is equivalent to the assignment _env.x = val, which in turn is equivalent to

settable_event(_env,"x",val)

where _env is the environment of the running function. (The _env variable is not defined in Idle. We use it here only for explanatory purposes.)

Control Structures
top of page

The control structures if, while, and repeat have the usual meaning and familiar syntax:

stat::=while exp do block end
	stat::=repeat block until exp
	stat::=if exp then block {elseif exp then block} [else block] end

Idle also has a for statement, in two flavors (see For Statement).

The condition expression of a control structure may return any value. Both false and nil are considered false. All values different from nil and false are considered true (in particular, the number 0 and the empty string are also true).

In the repeat–until loop, the inner block does not end at the until keyword, but only after the condition. So, the condition can refer to local variables declared inside the loop block.

The return statement is used to return values from a function or a chunk (which is just a function). Functions and chunks may return more than one value, so the syntax for the return statement is

stat::=return [explist1]

The break statement is used to terminate the execution of a while, repeat, or for loop, skipping to the next statement after the loop:

stat::=break

A break without argument ends the innermost enclosing loop. A break with a numerical argument of 0 continues the innermost enclosing loop. A break with a numerical argument greater 0 ends the respective enclosing loop. This means that break 1 and break are synonymous.

The return and break statements can only be written as the last statement of a block. If it is necessary to return or break in the middle of a block, then an explicit inner block should be used, as in the idioms do return end and do break end, because then return and break are the last statements in their (inner) blocks.

For Statement
top of page

The for statement has two forms: one numeric and one generic. The numeric for loop repeats a block of code while a control variable runs through an arithmetic progression. It has the following syntax:

stat::=for Name'='exp','exp[','exp] do block end

The block is repeated for name starting at the value of the first exp, until it passes the second exp by steps of the third exp. More precisely, a for statement like

for v=e1,e2,e3 do block end

is equivalent to the code:

do
    local var,limit,step=tonumber(e1),tonumber(e2),tonumber(e3)
    if not (var and limit and step) then error() end
    while (step>0 and var<=limit) or (step<=0 and var>=limit) do
      local v=var
      block
      var=var+step
    end
end

Note the following:

The generic for statement works over functions, called iterators. On each iteration, the iterator function is called to produce a new value, stopping when this new value is nil. The generic for loop has the following syntax:

stat::=for namelist in explist1 do block end
namelist::=Name{','Name}

A for statement like

for var_1,...,var_n in explist do block end

is equivalent to the code:

do
  local f,s,var=explist
  while true do
    local var_1,...,var_n=f(s,var)
    var=var_1
    if var==nil then break end
    block
  end
end

Note the following:

Function Calls as Statements
top of page

To allow possible side-effects, function calls can be executed as statements:

stat::=functioncall

In this case, all returned values are thrown away. Function calls are explained in Function Calls.

Local Declarations
top of page

Local variables may be declared anywhere inside a block. The declaration may include an initial assignment:

stat::=local namelist['='explist1]

If present, an initial assignment has the same semantics of a multiple assignment (see Assignment). Otherwise, all variables are initialised with nil.

A chunk is also a block (see Chunks), and so local variables can be declared in a chunk outside any explicit block. The scope of such local variables extends until the end of the chunk.

The visibility rules for local variables are explained in Visibility Rules.

Expressions
top of page

The basic expressions in Idle are the following:

exp::=prefixexp
exp::=nil|false|true
exp::=Number
exp::=String
exp::=function
exp::=tableconstructor
exp::='...'
exp::=exp binop exp
exp::=unop exp
prefixexp::=var|functioncall|'('exp')'

Numbers and literal strings are explained in Lexical Conventions; variables are explained in Variables; function definitions are explained in Function Definitions; function calls are explained in Function Calls; table constructors are explained in Table Constructors. Vararg expressions, denoted by three dots ('...'), can only be used when directly inside a vararg function; they are explained in Function Definitions.

Binary operators comprise arithmetic operators (see Arithmetic Operators), relational operators (see Relational Operators), logical operators (see Logical Operators), and the concatenation operator (see Concatenation). Unary operators comprise the unary minus (see Arithmetic Operators), the unary not (see Logical Operators), and the unary length operator (see The Length Operator).

Both function calls and vararg expressions may result in multiple values. If the expression is used as a statement (see Function Calls as Statements) (only possible for function calls), then its return list is adjusted to zero elements, thus discarding all returned values. If the expression is used as the last (or the only) element of a list of expressions, then no adjustment is made (unless the call is enclosed in parentheses). In all other contexts, Idle adjusts the result list to one element, discarding all values except the first one.

Here are some examples:

f()                -- adjusted to 0 results
g(f(),x)           -- f() is adjusted to 1 result
g(x,f())           -- g gets x plus all results from f()
a,b,c=f(),x        -- f() is adjusted to 1 result (c gets nil)
a,b=...            -- a gets the first vararg parameter, b gets
                   -- the second (both a and b may get nil if there
                   -- is no corresponding vararg parameter)
a,b,c=x,f()        -- f() is adjusted to 2 results
a,b,c=f()          -- f() is adjusted to 3 results
return f()         -- returns all results from f()
return ...         -- returns all received vararg parameters
return x,y,f()     -- returns x, y, and all results from f()
{f()}              -- creates a list with all results from f()
{...}              -- creates a list with all vararg parameters
{f(),nil}          -- f() is adjusted to 1 result

An expression enclosed in parentheses always results in only one value. Thus, (f(x,y,z)) is always a single value, even if f returns several values. (The value of (f(x,y,z)) is the first value returned by f or nil if f does not return any values.)

Arithmetic Operators
top of page

Idle supports the usual arithmetic operators: the binary + (addition), - (subtraction), * (multiplication), / (division), % (modulo), and ^ (exponentiation); and unary - (negation). If the operands are numbers, or strings that can be converted to numbers (see Coercion), then all operations have the usual meaning. Exponentiation works for any exponent. For instance, x^(-0.5) computes the inverse of the square root of x. Modulo is defined as

a%b==a-math.floor(a/b)*b

That is, it is the remainder of a division that rounds the quotient towards minus infinity.

Relational Operators
top of page

The relational operators in Idle are

==    ~=    !=    <     >     <=    >=

These operators always result in false or true.

Equality (==) first compares the type of its operands. If the types are different, then the result is false. Otherwise, the values of the operands are compared. Numbers and strings are compared in the usual way. Objects (tables, userdata, threads, and functions) are compared by reference: two objects are considered equal only if they are the same object. Every time you create a new object (a table, userdata, thread, or function), this new object is different from any previously existing object.

You can change the way that Idle compares tables and userdata by using the "eq" metamethod (see Metatables).

The conversion rules of Coercion do not apply to equality comparisons. Thus, "0"==0 evaluates to false, and t[0] and t["0"] denote different entries in a table.

The operator ~= and the synonymous != are exactly the negation of equality (==).

The order operators work as follows. If both arguments are numbers, then they are compared as such. Otherwise, if both arguments are strings, then their values are compared according to the current locale. Otherwise, Idle tries to call the "lt" or the "le" metamethod (see Metatables).

Logical Operators
top of page

The logical operators in Idle are and, or, and not. Like the control structures (see Control Structures), all logical operators consider both false and nil as false and anything else as true.

The negation operator not always returns false or true. The conjunction operator and returns its first argument if this value is false or nil; otherwise, and returns its second argument. The disjunction operator or returns its first argument if this value is different from nil and false; otherwise, or returns its second argument. Both and and or use short-cut evaluation; that is, the second operand is evaluated only if necessary. Here are some examples:

10 or 20            --> 10
10 or error()       --> 10
nil or "a"          --> "a"
nil and 10          --> nil
false and error()   --> false
false and nil       --> false
false or nil        --> nil
10 and 20           --> 20
--> indicates the result of the preceding expression.

Concatenation
top of page

The string concatenation operator in Idle is denoted by two dots ('..'). If both operands are strings or numbers, then they are converted to strings according to the rules mentioned in Coercion. Otherwise, the "concat" metamethod is called (see Metatables).

The Length Operator
top of page

The length operator is denoted by the unary operator #. The length of a string is its number of bytes (that is, the usual meaning of string length when each character is one byte).

The length of a table t is defined to be any integer index n such that t[n] is not nil and t[n+1] is nil; moreover, if t[1] is nil, n may be zero. For a regular array, with non-nil values from 1 to a given n, its length is exactly that n, the index of its last value. If the array has "holes" (that is, nil values between other non-nil values), then #t may be any of the indices that directly precedes a nil value (that is, it may consider any such nil value as the end of the array).

Precedence
top of page

Operator precedence in Idle follows the table below, from lower to higher priority:

or
and
<     >     <=    >=    ~=    !=    ==
..
+     -
*     /     %
not   #     - (unary)
^

As usual, you can use parentheses to change the precedences of an expression. The concatenation ('..') and exponentiation ('^') operators are right associative. All other binary operators are left associative.

Table Constructors
top of page

Table constructors are expressions that create tables. Every time a constructor is evaluated, a new table is created. Constructors can be used to create empty tables, or to create a table and initialise some of its fields. The general syntax for constructors is

tableconstructor::='{'[fieldlist]'}'
fieldlist::=field{fieldsep field} [fieldsep]
field::='['exp']''='exp|Name'='exp|exp
fieldsep::=','

Each field of the form [exp1]=exp2 adds to the new table an entry with key exp1 and value exp2. A field of the form name=exp is equivalent to ["name"]=exp. Finally, fields of the form exp are equivalent to [i]=exp, where i are consecutive numerical integers, starting with 1. Fields in the other formats do not affect this counting. For example,

a={[f(1)]=g;"x","y";x=1,f(x),[30]=23;45}

is equivalent to

do
  local t={}
  t[f(1)]=g
  t[1]="x"         -- 1st exp
  t[2]="y"         -- 2nd exp
  t.x=1            -- t["x"]=1
  t[3]=f(x)        -- 3rd exp
  t[30]=23
  t[4]=45          -- 4th exp
  a=t
end

If the last field in the list has the form exp and the expression is a function call or a vararg expression, then all values returned by this expression enter the list consecutively (see Function Calls). To avoid this, enclose the function call (or the vararg expression) in parentheses (see Expressions).

The field list may have an optional trailing separator, as a convenience for machine-generated code.

Function Calls
top of page

A function call in Idle has the following syntax:

functioncall::=prefixexp args

In a function call, first prefixexp and args are evaluated. If the value of prefixexp has type function, then this function is called with the given arguments. Otherwise, the prefixexp "call" metamethod is called, having as first parameter the value of prefixexp, followed by the original call arguments (see Metatables).

The form

functioncall::=prefixexp':'Name args

can be used to call "methods". A call v:name(args) is syntactic sugar for v.name(v,args), except that v is evaluated only once.

Arguments have the following syntax:

args::='('[explist1]')'
	args::=tableconstructor
	args::=String

All argument expressions are evaluated before the call. A call of the form f{fields} is syntactic sugar for f({fields}); that is, the argument list is a single new table. A call of the form f'string' (or f"string" or fstring) is syntactic sugar for f('string'); that is, the argument list is a single literal string.

As an exception to the free-format syntax of Idle, you cannot put a line break before the '(' in a function call. This restriction avoids some ambiguities in the language. If you write

a=f
(g).x(a)

Idle would see that as a single statement: a=f(g).x(a). So, if you want two statements, you must add a semi-colon between them. If you actually want to call f, you must remove the line break before (g).

A call of the form return functioncall is called a tail call. Idle implements proper tail calls (or proper tail recursion): in a tail call, the called function reuses the stack entry of the calling function. Therefore, there is no limit on the number of nested tail calls that a program can execute. However, a tail call erases any debug information about the calling function. Note that a tail call only happens with a particular syntax, where the return has one single function call as argument; this syntax makes the calling function return exactly the returns of the called function. So, none of the following examples are tail calls:

return (f(x))        -- results adjusted to 1
return 2*f(x)
return x,f(x)        -- additional results
f(x); return         -- results discarded
return x or f(x)     -- results adjusted to 1

Function Definitions
top of page

The syntax for function definition is

function::=function funcbody
funcbody::='('[parlist1]')' block end

The following syntactic sugar simplifies function definitions:

stat::=function funcname funcbody
stat::=local function Name funcbody
funcname::=Name{'.'Name}[':'Name]

The statement

function f() body end

translates to

f=function() body end

The statement

function t.a.b.c.f() body end

translates to

t.a.b.c.f=function() body end

The statement

local function f() body end

translates to

local f;f=function() body end

and not to

local f=function() body end

(This only makes a difference when the body of the function contains references to f.)

A function definition is an executable expression, whose value has type function. When Idle pre-compiles a chunk, all its function bodies are pre-compiled too. Then, whenever Idle executes the function definition, the function is instantiated (or closed). This function instance (or closure) is the final value of the expression. Different instances of the same function may refer to different external local variables and may have different environment tables.

Parameters act as local variables that are initialised with the argument values:

parlist1::=namelist[',''...']|'...'

When a function is called, the list of arguments is adjusted to the length of the list of parameters, unless the function is a variadic or vararg function, which is indicated by three dots ('...') at the end of its parameter list. A vararg function does not adjust its argument list; instead, it collects all extra arguments and supplies them to the function through a vararg expression, which is also written as three dots. The value of this expression is a list of all actual extra arguments, similar to a function with multiple results. If a vararg expression is used inside another expression or in the middle of a list of expressions, then its return list is adjusted to one element. If the expression is used as the last element of a list of expressions, then no adjustment is made (unless the call is enclosed in parentheses).

As an example, consider the following definitions:

function f(a,b) end
function g(a,b,...) end
function r() return 1,2,3 end

Then, we have the following mapping from arguments to parameters and to the vararg expression:

CALL            PARAMETERS
f(3)             a=3, b=nil
f(3,4)           a=3, b=4
f(3,4,5)         a=3, b=4
f(r(),10)        a=1, b=10
f(r())           a=1, b=2
g(3)             a=3, b=nil, ... -->  (nothing)
g(3,4)           a=3, b=4,   ... -->  (nothing)
g(3,4,5,8)       a=3, b=4,   ... -->  5  8
g(5,r())         a=5, b=1,   ... -->  2  3

Results are returned using the return statement (see Control Structures). If control reaches the end of a function without encountering a return statement, then the function returns with no results.

The colon syntax is used for defining methods, that is, functions that have an implicit extra parameter self. Thus, the statement

function t.a.b.c:f(params) body end

is syntactic sugar for

t.a.b.c.f=function(self,params) body end -- with self==t.a.b.c.f

Visibility Rules
top of page

Idle is a lexically scoped language. The scope of variables begins at the first statement after their declaration and lasts until the end of the innermost block that includes the declaration. Consider the following example:

x=10                  -- global variable
do                    -- new block
  local x = x         -- new 'x', with value 10
  print(x)            --> 10
  x=x+1
  do                  -- another block
    local x=x+1       -- another 'x'
    print(x)          --> 12
  end
  print(x)            --> 11
end
print(x)              --> 10  (the global one)

Notice that, in a declaration like local x=x, the new x being declared is not in scope yet, and so the second x refers to the outside variable.

Because of the lexical scoping rules, local variables can be freely accessed by functions defined inside their scope. A local variable used by an inner function is called an upvalue, or external local variable, inside the inner function.

Notice that each execution of a local statement defines new local variables (even within the same scope). Consider the following example:

a={}
local x=20
for i=1,10 do
  local y=0
  a[i]=function() y=y+1; return x+y end
end

The loop creates ten closures (that is, ten instances of the anonymous function). Each of these closures uses a different y variable, while all of them share the same x.

Error Handling
top of page

Because Idle is an embedded extension language, all Idle actions start from C code in the host program calling a function from the Idle library. Whenever an error occurs during Idle compilation or execution, control returns to C, which can take appropriate measures (such as printing an error message).

Idle code can explicitly generate an error by calling the error( ) function. If you need to catch errors in Idle, you can use the pcall( ) function.

Metatables
top of page

Every value in Idle may have a metatable. This metatable is an ordinary Idle table that defines the behavior of the original value under certain special operations. You can change several aspects of the behavior of operations over a value by setting specific fields in its metatable. For instance, when a non-numeric value is the operand of an addition, Idle checks for a function in the field "__add" in its metatable. If it finds one, Idle calls this function to perform the addition.

We call the keys in a metatable events and the values metamethods. In the previous example, the event is 'add' and the metamethod is the function that performs the addition.

You can query the metatable of any value through the getmetatable( ) function.

You can replace the metatable of tables through the setmetatable( ) function. You cannot change the metatable of other types from Idle (except using the debug library); you must use the C API for that.

Tables and userdata have individual metatables (although multiple tables and userdata can share their metatables); values of all other types share one single metatable per type. So, there is one single metatable for all numbers, and for all strings, etc.

A metatable may control how an object behaves in arithmetic operations, order comparisons, concatenation, length operation, and indexing. A metatable can also define a function to be called when a userdata is garbage collected. For each of these operations Idle associates a specific key called an event. When Idle performs one of these operations over a value, it checks whether this value has a metatable with the corresponding event. If so, the value associated with that key (the metamethod) controls how Idle will perform the operation.

Metatables control the operations listed next. Each operation is identified by its corresponding name. The key for each operation is a string with its name prefixed by two underscores, '__'; for instance, the key for operation 'add' is the string "__add". The semantics of these operations is better explained by an Idle function describing how the interpreter executes the operation.

The code shown here in Idle is only illustrative; the real behavior is hard coded in the interpreter and it is much more efficient than this simulation. All functions used in these descriptions (rawget( ), tonumber( ), et cetera) are described in the Idle Runtime library documentation. In particular, to retrieve the metamethod of a given object, we use the expression

metatable(obj)[event]

This should be read as

rawget(getmetatable(obj) or {},event)

That is, the access to a metamethod does not invoke other metamethods, and the access to objects with no metatables does not fail (it simply results in nil).

'add': the + operation.
top of page

The function getbinhandler below defines how Idle chooses a handler for a binary operation. First, Idle tries the first operand. If its type does not define a handler for the operation, then Idle tries the second operand.

function getbinhandler(op1,op2,event)
  return metatable(op1)[event] or metatable(op2)[event]
end

By using this function, the behavior of the op1+op2 is

function add_event(op1,op2)
  local o1,o2=tonumber(op1),tonumber(op2)
  if o1 and o2 then  -- both operands are numeric?
    return o1+o2     -- '+' here is the primitive 'add'
  else  -- at least one of the operands is not numeric
    local h=getbinhandler(op1,op2,"__add")
    if h then  -- call the handler with both operands
      return h(op1, op2)
    else  -- no handler available: default behavior
      error(...)
    end
  end
end

'sub': the - operation.
top of page

Behavior similar to the 'add' operation.

'mul': the * operation.
top of page

Behavior similar to the 'add' operation.

'div': the / operation.
top of page

Behavior similar to the 'add' operation.

'mod': the % operation.
top of page

Behavior similar to the 'add' operation, with the operation o1-floor(o1/o2)*o2 as the primitive operation.

'pow': the ^ (exponentiation) operation.
top of page

Behavior similar to the 'add' operation, with the function pow (from the C math library) as the primitive operation.

'unm': the unary - operation.
top of page

function unm_event(op)
  local o=tonumber(op)
  if o then    -- operand is numeric?
    return -o  -- '-' here is the primitive 'unm'
  else  -- the operand is not numeric.
    -- Try to get a handler from the operand
    local h=metatable(op).__unm
    if h then  -- call the handler with the operand
      return h(op)
    else  -- no handler available: default behavior
      error(...)
    end
  end
end

'concat': the .. (concatenation) operation.
top of page

function concat_event(op1,op2)
  if (type(op1)=="string" or type(op1)=="number") and
     (type(op2)=="string" or type(op2)=="number") then
    return op1..op2  -- primitive string concatenation
  else
    local h=getbinhandler(op1,op2,"__concat")
    if h then return h(op1,op2)
    else error(...)
    end
  end
end

'len': the # operation.
top of page

function len_event(op)
  if type(op)=="string" then
    return strlen(op)  -- primitive string length
  elseif type(op)=="table" then
    return #op  -- primitive table length
  else
    local h=metatable(op).__len
    if h then  -- call the handler with the operand
      return h(op)
    else  -- no handler available: default behavior
      error(...)
    end
  end
end

See The Length Operator for a description of the length of a table.

'eq': the == operation.
top of page

The function getcomphandler defines how Idle chooses a metamethod for comparison operators. A metamethod only is selected when both objects being compared have the same type and the same metamethod for the selected operation.

function getcomphandler(op1,op2,event)
  if type(op1)~=type(op2) then return nil end
  local mm1=metatable(op1)[event]
  local mm2=metatable(op2)[event]
  if mm1==mm2 then return mm1 else return nil end
end

The 'eq' event is defined as follows:

function eq_event(op1,op2)
  if type(op1)~=type(op2) then  -- different types?
    return false  -- different objects
  end
  if op1==op2 then  -- primitive equal?
    return true     -- objects are equal
  end
  -- try metamethod
  local h=getcomphandler(op1,op2,"__eq")
  if h then return h(op1, op2)
  else return false end
end

a~=b is equivalent to not (a==b).

'lt': the < operation.
top of page

function lt_event(op1,op2)
  if type(op1)=="number" and type(op2)=="number" then
    return op1<op2  -- numeric comparison
  elseif type(op1)=="string" and type(op2)=="string" then
    return op1<op2  -- lexicographic comparison
  else
    local h=getcomphandler(op1,op2,"__lt")
    if h then return h(op1, op2)
    else error(···) end
  end
end

a > b is equivalent to b < a.

'le': the <= operation.
top of page

function le_event(op1,op2)
  if type(op1)=="number" and type(op2)=="number" then
    return op1<=op2  -- numeric comparison
  elseif type(op1)=="string" and type(op2)=="string" then
    return op1<=op2  -- lexicographic comparison
  else
    local h=getcomphandler(op1,op2,"__le")
    if h then return h(op1,op2)
    else
      h=getcomphandler(op1,op2,"__lt")
      if h then return not h(op2,op1)
      else error(...) end
    end
  end
end

a >= b is equivalent to b <= a. Note that, in the absence of a 'le' metamethod, Idle tries the 'lt', assuming that a <= b is equivalent to not (b < a)

'index': The indexing access table[key].
top of page

This metamethod is only called if table[key] does not yet exist.

function gettable_event(table,key)
  local h
  if type(table)=="table" then
    local v=rawget(table,key)
    if v~=nil then return v end
    h=metatable(table).__index
    if h==nil then return nil end
  else
    h=metatable(table).__index
    if h==nil then error(...) end
  end
  if type(h)=="function" then
    return h(table,key)  -- call the handler
  else
    return h[key]        -- or repeat operation on it
  end
end

'newindex': The indexing assignment table[key]=value.
top of page

This metamethod is only called if table[key] does not yet exist.

function settable_event (table,key,value)
  local h
  if type(table)=="table" then
    local v=rawget(table,key)
    if v~=nil then rawset(table,key,value); return end
    h=metatable(table).__newindex
    if h==nil then rawset(table,key,value); return end
  else
    h=metatable(table).__newindex
    if h == nil then error(...) end
  end
  if type(h)=="function" then
    return h(table,key,value)  -- call the handler
  else
    h[key]=value               -- or repeat operation on it
  end
end

'usedindex': The indexing access assignment table[key]=value.
top of page

The metamethod __usedindex works like 'newindex' but it is called if table[key] already exists.

'call': called when Idle calls a value.
top of page

function function_event(func,...)
  if type(func)=="function" then
    return func(...)  -- primitive call
  else
    local h=metatable(func).__call
    if h then return h(func,...)
    else error(...) end
  end
end

'ipairs': called when the ipairs() function is called.
top of page

function ipairs_event(table)
  local h=metatable(table).__ipairs
  if h then return h(table)
  else return rawipairs(table)  -- default raw function
end

'next': called when the next() function is called.
top of page

function next_event(table,key)
  local h=metatable(table).__next
  if h then return h(table,key)
  else return rawnext(table,key)  -- default raw function
end

'pairs': called when the pairs() function is called.
top of page

function pairs_event(table)
  local h=metatable(table).__pairs
  if h then return h(table)
  else return rawpairs(table)  -- default raw function
end

'type': called when the type() function is called.
top of page

function type_event(val)
  local h=metatable(val).__type
  if h then return h(cal)
  else return rawtype(val)  -- default raw function
end

Environments
top of page

Besides metatables, objects of types thread, function, and userdata have another table associated with them, called their environment. Like metatables, environments are regular tables and multiple objects can share the same environment.

Environments associated with userdata have no meaning for Idle. It is only a convenience feature for programmers to associate a table to a userdata.

Environments associated with threads are called global environments. They are used as the default environment for their threads and non-nested functions created by the thread (through loadfile( ), loadstring( ) or load( )) and can be directly accessed by C code.

Environments associated with C functions can be directly accessed by C code. They are used as the default environment for other C functions created by the function.

Environments associated with Idle functions are used to resolve all accesses to global variables within the function (see Variables). They are used as the default environment for other Idle functions created by the function.

You can change the environment of an Idle function or the running thread by calling setfenv. You can get the environment of an Idle function or the running thread by calling getfenv. To manipulate the environment of other objects (userdata, C functions, other threads) you must use the C API.

Garbage Collection
top of page

Idle performs automatic memory management. This means that you have to worry neither about allocating memory for new objects nor about freeing it when the objects are no longer needed. Idle manages memory automatically by running a garbage collector from time to time to collect all dead objects (that is, these objects that are no longer accessible from Idle). All objects in Idle are subject to automatic management: tables, userdata, functions, threads, and strings.

Idle implements an incremental mark-and-sweep collector. It uses two numbers to control its garbage-collection cycles: the garbage-collector pause and the garbage-collector step multiplier.

The garbage-collector pause controls how long the collector waits before starting a new cycle. Larger values make the collector less aggressive. Values smaller than 1 mean the collector will not wait to start a new cycle. A value of 2 means that the collector waits for the total memory in use to double before starting a new cycle.

The step multiplier controls the relative speed of the collector relative to memory allocation. Larger values make the collector more aggressive but also increase the size of each incremental step. Values smaller than 1 make the collector too slow and may result in the collector never finishing a cycle. The default, 2, means that the collector runs at "twice" the speed of memory allocation.

You can change these numbers by calling collectgarbage( ) in Idle. This takes percentage points as arguments (so an argument of 100 means a real value of 1). With this functions you can also control the collector directly (e.g., stop and restart it).

Garbage-Collection Metamethods
top of page

Using the C API, you can set garbage-collector metamethods for userdata (see Metatables). These metamethods are also called finalisers. Finalisers allow you to coordinate Idle's garbage collection with external resource management (such as closing files, network or database connections, or freeing your own memory).

Garbage userdata with a field __gc in their metatables are not collected immediately by the garbage collector. Instead, Idle puts them in a list. After the collection, Idle does the equivalent of the following function for each userdata in that list:

function gc_event(udata)
  local h=metatable(udata).__gc
  if h then h(udata) end
end

At the end of each garbage-collection cycle, the finalisers for userdata are called in reverse order of their creation, among those collected in that cycle. That is, the first finaliser to be called is the one associated with the userdata created last in the program.

Weak Tables
top of page

A weak table is a table whose elements are weak references. A weak reference is ignored by the garbage collector. In other words, if the only references to an object are weak references, then the garbage collector will collect this object.

A weak table can have weak keys, weak values, or both. A table with weak keys allows the collection of its keys, but prevents the collection of its values. A table with both weak keys and weak values allows the collection of both keys and values. In any case, if either the key or the value is collected, the whole pair is removed from the table. The weakness of a table is controlled by the __mode field of its metatable. If the __mode field is a string containing the character 'k', the keys in the table are weak. If __mode contains 'v', the values in the table are weak.

After you use a table as a metatable, you should not change the value of its field __mode. Otherwise, the weak behavior of the tables controlled by this metatable is undefined.

Coroutines
top of page

Idle supports coroutines, also called collaborative multithreading. A coroutine in Idle represents an independent thread of execution. Unlike threads in multithread systems, however, a coroutine only suspends its execution by explicitly calling a yield function.

You create a coroutine with a call to coroutine.create( ). Its sole argument is a function that is the main function of the coroutine. The create function only creates a new coroutine and returns a handle to it (an object of type thread); it does not start the coroutine execution.

When you first call coroutine.resume( ), passing as its first argument the thread returned by coroutine.create( ), the coroutine starts its execution, at the first line of its main function. Extra arguments passed to coroutine.resume( ) are passed on to the coroutine main function. After the coroutine starts running, it runs until it terminates or yields.

A coroutine can terminate its execution in two ways: normally, when its main function returns (explicitly or implicitly, after the last instruction); and abnormally, if there is an unprotected error. In the first case, coroutine.resume( ) returns true, plus any values returned by the coroutine main function. In case of errors, coroutine.resume( ) returns false plus an error message.

A coroutine yields by calling coroutine.yield( ). When a coroutine yields, the corresponding coroutine.resume( ) returns immediately, even if the yield happens inside nested function calls (that is, not in the main function, but in a function directly or indirectly called by the main function). In the case of a yield, coroutine.resume( ) also returns true, plus any values passed to coroutine.yield( ). The next time you resume the same coroutine, it continues its execution from the point where it yielded, with the call to coroutine.yield( ) returning any extra arguments passed to coroutine.resume( ).

Like coroutine.create( ), the coroutine.wrap( ) function also creates a coroutine, but instead of returning the coroutine itself, it returns a function that, when called, resumes the coroutine. Any arguments passed to this function go as extra arguments to coroutine.resume( ). coroutine.wrap( ) returns all the values returned by coroutine.resume( ), except the first one (the boolean error code). Unlike coroutine.resume( ), coroutine.wrap does not catch errors; any error is propagated to the caller.

As an example, consider the following code:

function foo(a)
  print("foo",a)
  return coroutine.yield(2*a)
end

co=coroutine.create(function(a,b)
  print("co-body",a,b)
  local r=foo(a+1)
  print("co-body",r)
  local r,s=coroutine.yield(a+b,a-b)
  print("co-body",r,s)
  return b,"end"
end)
print("main",coroutine.resume(co,1, 10))
print("main",coroutine.resume(co,"r"))
print("main",coroutine.resume(co,"x","y"))
print("main",coroutine.resume(co,"x","y"))

When you run it, it will produce the following output:

co-body 1       10
foo     2

main    true    4
co-body r
main    true    11      -9
co-body x       y
main    true    10      end
main    false   cannot resume dead coroutine

Differences to Lua
top of page

This is a brief and informal description of the language differences between Idle and Lua 5.1. Most things are either minor points or mostly interesting in special circumstances. There are, however, a few rather important points to keep in mind.

LUA_PATH     ->   IDLE_PATH
LUA_CPATH    ->   IDLE_CPATH
LUA_INIT     ->   IDLE_INIT
LUA_LDIR          "!\\"
LUA_CDIR          "!\\"
LUA_PATH_DEFAULT  ".\\?.idle;..\\?.idle;"LUA_LDIR"?.idle;"LUA_LDIR"?\\init.idle"
LUA_CPATH_DEFAULT ".\\?.dll;..\\?.dll;"LUA_CDIR"?.dll;"LUA_CDIR"loadall.dll"
LUA_PROMPT        "Idle> "
LUA_PROMPT2       "Idle>> "
LUA_PROGNAME      "Idle"
LUA_MAXINPUT      1024
#undef LUA_COMPAT_VARARG
#undef LUA_COMPAT_MOD
#undef LUA_COMPAT_GFIND
#undef LUA_COMPAT_OPENLIB
LUAI_MAXCALLS     131072
LUAI_MAXCSTACK    8192
LUAI_MAXCCALLS    248
LUAI_MAXVARS      248
LUAI_MAXUPVALUES  248
LUAL_BUFFERSIZE   1024

For changes in the Idle compiler command line options see the Idle Tools page.

The Syntax of Idle
top of page

Here is the complete syntax of Idle in extended BNF. (It does not describe operator precedences.)

chunk ::= {stat [';']} [laststat [';']]
block ::= chunk
stat ::=  varlist1 '=' explist1 |
   functioncall |
   do block end |
   while exp do block end |
   repeat block until exp |
   if exp then block {elseif exp then block} [else block] end |
   for Name '=' exp ',' exp [',' exp] do block end |
   for namelist in explist1 do block end |
   function funcname funcbody |
   local function Name funcbody |
   local namelist ['=' explist1]
laststat ::= return [explist1] | break [Number]
funcname ::= Name {'.' Name} [':' Name]
varlist1 ::= var {',' var}
var ::=  Name | prefixexp '[' exp ']' | prefixexp '.' Name
namelist ::= Name {',' Name}
explist1 ::= {exp ','} exp
exp ::=  nil | false | true | Number | String | '...' | function |
  prefixexp | tableconstructor | exp binop exp | unop exp
prefixexp ::= var | functioncall | '(' exp ')'
functioncall ::=  prefixexp args | prefixexp ':' Name args
args ::=  '(' [explist1] ')' | tableconstructor | String
function ::= function funcbody
funcbody ::= '(' [parlist1] ')' block end
parlist1 ::= namelist [',' '...'] | '...'
tableconstructor ::= '{' [fieldlist] '}'
fieldlist ::= field {fieldsep field} [fieldsep]
field ::= '[' exp ']' '=' exp | Name '=' exp | exp
fieldsep ::= ',' | ';'
binop ::= '+' | '-' | '*' | '/' | '^' | '%' | '..' |
   '<' | '<=' | '>' | '>=' | '==' | '~=' | '!=' |
   and | or
unop ::= '-' | not | '#'


$$ built from IdleLanguage.txt d106963c4f77 Mon Sep 27 13:27:10 2010 +0000 thomasl $$