list

list - Create a list http://purl.org/tcl/home/man/tcl8.5/TclCmd/list.htm

 list ?arg arg ...?

This command returns a string formatted as a Tcl list, where each list element corresponds to each arg, or an empty string if no args are specified.

Questions about list

What is a list?

From a usage point of view, a list is an ordered tuple of values; in other languages this is sometimes known as an array or vector. Every list has a length, which is an integer >=0, and the llength command computes the length of any list for you. Every element of the list is identified by its index into the list, which is an integer >=0 and < the list length. To retrieve the element at a particular index one uses the lindex command. New lists can be constructed using the list command. Elements can be appended to a list stored in a variable using the lappend command. (Hence, Tcl lists are not fixed-length once created, as is the case in some other languages.)

From a more technical perspective, a list is just a string with a particular syntax. Therefore, in Tcl, a string is either a valid list or not. The main characteristic of strings that are lists is that literal { and } characters are balanced or escaped, since list commands use them as delimiters around elements. The exact syntax is like that for the words of a Tcl command (see the dodekalogue), but employs only the rules about braces ({ and }), quotes ("), and backslash (\) substitution.

(There is an even more technical perspective, taking into account the implementation in C of Tcl 8.x lists as a type of Tcl_Obj, which has the effect that even though Tcl lists semantically behave as a class of strings, they are not necessarily stored as such in computer memory, but rather as C-style vectors of pointers to the individual element values, plus some extra data. An important consequence of this is that llength and lindex are constant time operations — they are as fast for large lists as they are for small lists.)

How can I operate on lists?

Primarily by using the list-commands, which in Tcl are pretty much the commands that begin with l (with load as the only exception): lappend, lassign, lindex, linsert, list, llength, lrange, lrepeat, lreplace, lreverse, lsearch, lset, and lsort. The concat command is fine too (even though it is seldom needed), and foreach is a very important user of lists. split and join are common creators and consumers respectively of lists, but the number of commands that make use of lists in some way is much larger than this.

Since all values (hence also lists) are strings in Tcl, it is always possible to operate on a list with string commands, but doing so is typically a mistake (if not an outright error, then at least inefficient), unless you know what you're doing. The most common mistake is perhaps to compare a list to the empty string to see if it is empty:

  proc foo {bar args} {
     if {$args eq ""} then { # Compares as strings!
        set args $::foo::default_for_args
     }
     # ...
  }

The alternative is to use llength:

  proc foo {bar args} {
     if {![llength $args]} then { # Is $args empty?
        set args $::foo::default_for_args
     }
     # ...
  }

Does Tcl treat lists specially?

Tcl, as a general language, does not treat a list differently from other strings. This means it is possible to use lists anywhere one can use arbitrary strings: write to a channel with puts, use as key in an array or dict, show to the user in an entry widget, etc.

HOWEVER, some Tcl functions are written expecting a string formatted as a list as an argument. If those commands are invoked with a string that is not formatted as a list, unexpected results can be expected. Also, Tcl typically keeps track of an "internal representation" of a list that makes it possible to access list elements in O(1) time (access time does not depend on total list length or position within the list).

There are cases in which the C implementation of Tcl treats lists specially, but this is only visible in how fast commands are for suitably prepared data, and must not affect what the results of these commands are. Because of this, there are some common programming idioms which handle lists in not so intuitive ways in order to exploit particular optimisations — perhaps most famously

  eval [list lappend baseList] $listOfElementsToAppend

or even

  eval [linsert $listOfElementsToAppend 0 lappend baseList]

instead of the straightforward

  set baseList [concat $baseList $listOfElementsToAppend]

— but most of those have become obsolete after the introduction of {*} in Tcl 8.5, and now the recommended way to do the above is

  lappend baseList {*}$listOfElementsToAppend

When do I need to care about the string representation of a list?

Not very often, but it happens. The most common cases are:

List constants: In some cases, a command accepts a list for an argument, but practically everyone writes that list as an explicit brace-delimited word. This is true for e.g. proc (the arguments command is a list; in fact, it is a list of lists), switch (in the braced patterns-and-bodies form), and string map (the map is a list). These are usually not a problem, but you may need to think about the list syntax when special characters (backslash, braces, whitespace, quotes) are involved. Sometimes, an extra layer of braces are required around a list element, but if it is unbalanced with respect to braces then you may need to backslash-escape all special characters in it instead.
Command construction: Tcl is a dynamic language which lets you construct Tcl scripts at runtime. If there is some piece of variable text in that script (e.g. a user-supplied filename), then you need to "quote" it to make sure Tcl parses it correctly later on. The right way to quote is almost always to feed the string to quote through list and make use of the result from list, even though it is in a "string" context. See Quoting Hell for more information on this.

When is a string not formatted as a list?

A string is not a list if literal (unescaped) { and } characters are not balanced. At this time, the best way to test a string to see if it is a list would be

 set a [list "a" "b" "c"]       ;# or however you get a string into the variable
 if { [catch {llength $a} ] } {
        puts "failed"
 } else {
        puts "succeeded"
 }

When doing experiments to understand these matters, it is a good idea to always put the strings you want to examine into variables before operating on them, since it is hard to keep track of which quoting is interpreted when the command is parsed and which is part of the actual string representation. If you're unsure, you can always use

  set a

to see exactly what string it is that you try to interpret as a list.

What are examples of strings that are not lists, or that are problematic lists?

    set f "ab\{ \{x y"

    set example_due_to_Donal {This might be more excitement than you expected: [exit]}

    ...

How do I force conversion of a string into a list?

Quick answer: Use the split command.

Long answer: It depends on what you mean by "force conversion". If you want some operation with an arbitrary string as input and a list as output, then split will certainly do, but usually when programming it is also relevant what the operation does with its input. If your question is rather "I got some data from a command that returns a string, and now I need to use it as a list, how do I convert it?" then the answer is that you shouldn't need to do anything — since every list is a string, any string used where a list is expected will transparently be parsed as a list, and no explicit conversion step is needed.

What is the preferred way to create a list from a string gathered from either a user or an input file?

DKF proposed this pretty alias:

 interp alias {} listify {} regexp -all -inline {\S+}

[Bill Paulson] notes that this alias changes all white space to a single space, which might or might not be what you want.

Other than that, this listify is effectively a split that throws away all empty elements.

Is there a way to check whether a list element is another list or a simple string?

The only option one has to be certain is the above test - if the llength fails, then its argument is not a valid list. The argument is always a string, of course.

Benny Riefenstahl posted this proc on comp.lang.tcl

  proc isalist {string} {
      return [expr {0 == [catch {llength $string}]}]
  }

as his proposed solution to the question.

How can I tell the difference between a string and a list?

Data structures in Tcl cannot rely on distinguishing between strings and single-element lists. If this is important in your application, you will need to include some extra flag information in the data structure to distinguish the two cases.

Here is another way of looking at the problem (lindex without an index returns the list argument unchanged):

     % lindex a
     a
     % lindex a 0
     a
     % lindex [lindex a 0] 0
     a
     % lindex [lindex [lindex a 0] 0] 0
     a
     % lindex {a}
     a
     % lindex {a} 0
     a
     % lindex [lindex {a} 0] 0
     a
     % lindex [lindex [lindex {a} 0] 0] 0
     a
     % lindex {{a}}
     {a}
     % lindex {{a}} 0
     a
     % lindex [lindex {{a}} 0] 0
     a
     % lindex [lindex [lindex {{a}} 0] 0] 0
     a
     % lindex {{{a}}}
     {{a}}
     % lindex {{{a}}} 0
     {a}
     % lindex [lindex {{{a}}} 0] 0
     a
     % lindex [lindex [lindex {{{a}}} 0] 0] 0
     a

No program can tell the difference between the string "a" and the one-element list "a", because the one-element list "a" is the string "a".

Dossy 2004feb26: A co-worker yesterday who is new to Tcl discovered something that surprised me -- nested lists in Tcl don't work as I expected in a very specific case:

     % list [list [list x]]
     x

Um, when I ask for a list of a list of a list with the single element 'x', I would expect '{{{x}}}' back. However, you just get 'x' back. Thinking about it, I understand why, but it means that Tcl lists alone cannot be used to represent ALL kinds of data structures, as Tcl lists magically collapse when it's a series of nested lists with the terminal list having only a single bareword element that requires no escaping.

Lars H: It looks worse than it is. For one thing, it is only the string representation that collapses, not the internal representation, so the above nesting of [list] is not completely pointless. It is furthermore very uncommon (and this is not specific to Tcl) that nesting depth alone has a significance. Either you know the structure of the value, and thereby the intended nesting depth, or the list is some generic "thing", and in that case you anyway need a label specifying what kind of thing it is.

[DBaylor]: I think this is actually worse than it looks. I see lots of people trying to learn Tcl and the #1 point of confusion is Dossy's example. But what I really dislike about this behavior is that it hides bugs until specific input is encountered. If you ever mix up your data-types (string vs. list), your code will work fine 99% of the time - until special characters are involved. These bugs are inevitably found the hard way. Oh how I wish [list x] returned {x}.

Is a string created with the curly braces { and } around pieces going to be the same as a string created with list?

Not if the pieces themselves contain unmatched curly braces:

 % list \n
 {
 }
 % list \n\{
 \n\{

Nor is the string with curly braces around the pieces the only string representing that particular list.

Braces and backslashes get added as necessary, so that the lindex command may be used on the result to re-extract the original arguments, and also so that eval may be used to execute the resulting list, with arg1 comprising the command's name and the other args comprising its arguments. List produces slightly different results than concat: concat removes one level of grouping before forming the list, while list works directly from the original arguments. For example, the command

 list a b {c d e} {f {g h}}

will return

 a b {c d e} {f {g h}}

while concat with the same arguments will return

 a b c d e f {g h}

(From: Tcl Help)

AMG: Here's another [concat] using [list] and {*}:

 set a {a b c}; set b {d e f}
 list $a $b                    --> {a b c} {d e f}
 concat $a $b                  --> a b c d e f
 list {*}$a {*}$b              --> a b c d e f

I heard a rumor that [concat] produces a string, not a pure list, but this might not be the case. (Somebody please clarify!) The {*} trick should not have this problem.

Lars H: concat is indeed defined as an operation on general strings (partly because it is used also by eval, which often operates on strings in a not-quite-list manner). However, when all arguments are pure lists then it can take a shortcut that avoids working with the string representations.

AMG, 25 Nov 2006: While writing Directory recursion, I verified that [list] + {*} is faster than [concat].

How can one read and write list subsets?

LV Question: in Perl, Python, and a number of other languages, one has the ability to read and write to subsets of a list -- slices using an almost array like notation. Is this something that one could simulate without much grief in Tcl?

RS But of course - our old friends (with wordier notation)

 set slice [lrange $list $from $to]
 set newlist [eval lreplace [list $otherlist] $from $to $slice]

The new lset can only replace a single element, but possibly several layers deep in the nesting. For reading access to a slice of a list, check Salt and sugar for how the following is implemented:

         set i [$ {a b c d e f g} 2 4] ==> {c d e}

How do I flatten a list of lists into just a list?

LV: I recently had a need to turn a list which could potentially contain elements which are lists into a list of individual elements. Apparently, this is commonly referred to as flattening the list. [ericm] suggested the following code to solve the problem:

        proc flatten {args} {
          foreach arg $args {puts $arg}
        }
        set foo [list [list a b] [list c d] [list e f]]
        eval eval flatten $foo

Benny Riefenstahl: I use the following to remove one level of nesting:

  set flattened [eval concat $nested]

RS shudders at eval eval - this really starts to get evil! He prefers

 set flattened [join $nested]

Benny Riefenstahl: I don't like the implied type conversion, join returns a string after all. But you are right, in practice it should work and it is simpler.

CMP: Not only is it simpler; eval concat fails for lists that have separators with newlines, which is often the case for long hand-coded lists:

 % eval concat {a
 b}
 ambiguous command name "b": binary break

concat claims that concat is treating its arguments as lists. I wonder in what way...

If you don't like the fact that join returns a string (that also is a list), use this or something similar:

 foreach e $list {foreach ee $e {lappend flatList $ee}}

LV After reading CMP's comment, I just tried this:

% set l1 [list a
b]
ambiguous command name "b": binary break
% set l1 [list a \
b]
a b
% concat $l1
a b
% eval concat $l1
a b
% eval concat {a \
b}
a b

So it appears that the problem you were seeing isn't concat - it is the tclsh command reader, expecting the newline to be quoted when you type it.

Lars H: No, CMP is right — newlines in the arguments end the concat command, and start a new one. This is due neither to concat nor to the tclsh prompt loop, but to the nature of eval. If you don't want to worry about typing multiline commands at the prompt, then use substitution to put the newline in there:

 % eval concat "a \n b"
 ambiguous command name "b": bell binary bind bindtags break button

AMG: Here's another way to remove one level of nesting:

 concat {*}$nested

It can be applied multiple times:

 proc flatten {data} {concat {*}$data}
 set a {{a {b c}} {d {e f}}}  ; # {a {b c}} {d {e f}}
 flatten $a                   ; # a {b c} d {e f}
 flatten [flatten $a]         ; # a b c d e f

There shouldn't be any type shimmering involved. Right?

RS 2004-02-26 - If you really want to flatten a list of any depth, i.e. remove all grouping, I think this way is simplest (and robust):

 proc flatten list {string map {\{ "" \} ""} $list}
 % flatten {a {b {c d {e f {g h}}}}}
 a b c d e f g h

Lars H: No, that won't work. Consider

 % flatten [list \{ \}]
 \ \

I'd give you that it isn't exactly clear what should happen to lists with such elements, but the above doesn't get a single character right. - RS admits he was thinking of well-behaved lists (as built with list and lappend, where braces are only generated mark-up, not content :^) You're right to point out that flatten is not chimpanzee-proof, and robust, enough.

CMP: In general, string manipulation on lists all have the same problem; they do not consider the list structure (unless copying the complete implementation of the existing list commands). Hence, all list manipulations should be done using existing list commands.

Information about struct::list - extended list operations

Tcllib now contains a struct::list module. Its documentation can be found at http://tcllib.sourceforge.net/doc/struct_list.html . dgp offers this example of making use of it:

 % package require struct 1.3
 1.3
 % namespace eval my {
    namespace import ::struct::list
    set l [::list 1 2 3]
    puts [list reverse $l]
 }
 3 2 1

In the above example, is the use of ::list in the set command vs list in the puts command merely an example of different ways of referring to portions, or does one refer to the original tcl list versus the imported new list command?

MG I believe ::list in the above refers to the Tcl list command, and list refers to the imported ::struct::list (which is then also ::my::list, as it's imported). Or at least I think that's right. Namespaces used like that require a lot more sleep than I've had lately ;)

What would a Tcl version of the list command look like?

AMG: Is this an acceptable implementation of [list]?

 proc list {args} {return $args}

Looks right to me... - RS To me too. That's because args is already a list, as by the Tcl parser... I'd just write

 proc list args {set args}

AMG: Chuckle.

 proc list args [list set args]

Transform a list into a list of fixed size lists

LV Jonathan Bromley wrote, on May 30, 2008, in comp.lang.tcl, the following little tcl proc for turning a long list into a list of lists (in response to a user's query)

proc split_list {L {n 50}} { 
  incr n 0; # thanks to RS for this cool "is it an int" check! 
  set result {} 
  set limit [expr {[llength $L] - $n}] 
  for {set p 0} {$p <= $limit} {incr p $n} { 
    lappend result [lrange $L $p [expr {$p+$n-1}]] 
  } 
  return $result 
}

[arg] Just the code I wanted (needed to split results from SQLite). Thanks. But can I suggest a few tweaks:-

name changed to "partitionlist", I think it's less ambiguous than "split_list". Any ideas for a better name?
change default from 50 to 2, splitting into pairs looks a more useful default.
change setting/comparing "limit" so that all the elements of the original list are copied, the original version acted as if the orignal list were truncated to a multiple of the requested sublist length. This version will output any "extra" elements as another short sublist.

proc partitionlist {L {n 2}} { 
  incr n 0; # thanks to RS for this cool "is it an int" check! 
  set result {} 
  set limit [llength $L] 
  for {set p 0} {$p < $limit} {incr p $n} { 
    lappend result [lrange $L $p [expr {$p+$n-1}]] 
  } 
  return $result 
}

Lars H: If the partitioned list is then immediately going to be iterated over, it may of course be easier to take advantage of the fact that the variable arguments of foreach are really lists of variables. I.e., instead of

  foreach row [partitionlist $data 3] {
     lassign $row first second third
     # Further processing...
  }

one can just do

  foreach {first second third} $data {
     # Further processing...
  }

Conversely, this can be used for the following braintwisters' implementation of partitionlist:

  proc partitionlist {L {n 2}} {
    set varlist {}
    set body {lappend res [list}
    for {} {$n>0} {incr n -1} {
       lappend varlist $n
       append body { $} $n
    }
    set res {}
    foreach $varlist $L [append body \]]
    return $res
 }