32.4. Socket Streams

32.4. Socket Streams
Prev	Chapter 32. Platform Specific Extensions	Next

Platform Dependent: UNIX, Win32 platforms only.

32.4.1. Introduction
32.4.2. Socket API Reference
32.4.3. Argument :TIMEOUT

List of Examples

32.13. Lisp read-eval-print loop server
32.14. Lisp HTTP client

32.4.1. Introduction

List of Examples

32.13. Lisp read-eval-print loop server
32.14. Lisp HTTP client

Sockets are used for interprocess communications by processes running on the same host as well as by processes running on different hosts over a computer network. The most common kind of sockets is Internet stream sockets, and a high-level interface to them is described here. A more low level interface that closely follows the C system calls is also available, see Section 33.17, “Raw Socket Access”.

Two main varieties of sockets are interfaced to:

“active” sockets: correspond to SOCKET:SOCKET-STREAMs which are bidirectional STREAMs
“passive” sockets: correspond to SOCKET:SOCKET-SERVERs which are a special kind of objects that are used to allow the other side to initiate interaction with lisp.

Example 32.13. Lisp read-eval-print loop server

Here is a simple lisp read-eval-print loop server that waits for a remote connection and evaluates forms read from it:

(LET ((server (SOCKET:SOCKET-SERVER)))
  (FORMAT t "~&Waiting for a connection on ~S:~D~%"
          (SOCKET:SOCKET-SERVER-HOST server) (SOCKET:SOCKET-SERVER-PORT server))
  (UNWIND-PROTECT
      ;; infinite loop, terminate with Control+C
      (LOOP (WITH-OPEN-STREAM (socket (SOCKET:SOCKET-ACCEPT server))
              (MULTIPLE-VALUE-BIND (local-host local-port) (SOCKET:SOCKET-STREAM-LOCAL socket)
                (MULTIPLE-VALUE-BIND (remote-host remote-port) (SOCKET:SOCKET-STREAM-PEER socket)
                  (FORMAT T "~&Connection: ~S:~D -- ~S:~D~%"
                          remote-host remote-port local-host local-port)))
              ;; loop is terminated when the remote host closes the connection or on EXT:EXIT
              (LOOP (WHEN (EQ :eof (SOCKET:SOCKET-STATUS (cons socket :input))) (RETURN))
                    (PRINT (EVAL (READ socket)) socket)
                    ;; flush everything left in socket
                    (LOOP :for c = (READ-CHAR-NO-HANG socket nil nil) :while c)
                    (TERPRI socket))))
    ;; make sure server is closed
    (SOCKET:SOCKET-SERVER-CLOSE server)))

This opens a gaping security hole!

Functions like EXT:SHELL, EXT:EXECUTE, EXT:RUN-SHELL-COMMAND will allow the remote host to execute arbitrary code with your permissions. While functions defined in lisp (like EXT:RUN-SHELL-COMMAND) can be removed (using FMAKUNBOUND), the built-in functions (like EXT:SHELL and EXT:EXECUTE) cannot be permanently removed from the runtime, and an experienced hacker will be able to invoke them even if you FMAKUNBOUND their names.

You should limit the socket server to local connections by passing the STRING "127.0.0.1" as the :INTERFACE argument to SOCKET:SOCKET-SERVER.

Example 32.14. Lisp HTTP client

Here are a couple of simple lisp HTTP clients that fetch a web page and a binary file, and upload a file:

(DEFUN wget-text (host page file &OPTIONAL (port 80))
  ;; HTTP requires the :DOS line terminator
  (WITH-OPEN-STREAM (socket (SOCKET:SOCKET-CONNECT port host :EXTERNAL-FORMAT :DOS))
     (FORMAT socket "GET ~A HTTP/1.0~2%" page)
     ;; dump the whole thing - header+data - into the output file
     (WITH-OPEN-FILE (out file :direction :output)
       (LOOP :for line = (READ-LINE socket nil nil) :while line
          :do (WRITE-LINE line out)))))
(DEFUN wget-binary (host page file &OPTIONAL (port 80))
  (WITH-OPEN-STREAM (socket (SOCKET:SOCKET-CONNECT port host :EXTERNAL-FORMAT :DOS))
    (FORMAT socket "GET ~A HTTP/1.0~2%" page)
    (LOOP :with content-length :for line = (READ-LINE socket nil nil)
      ;; header is separated from the data with a blank line
      :until (ZEROP (LENGTH line)) :do
      (WHEN (STRING= line #1="Content-length: " :end1 #2=#.(LENGTH #1#))
        (SETQ content-length (PARSE-INTEGER line :start #2#))
      ;; this will not work if the server does not supply the content-length header
      :finally (RETURN (LET ((data (MAKE-ARRAY content-length
                                               :element-type '(UNSIGNED-BYTE 8))))
                           ;; switch to binary i/o on socket
                           (SETF (STREAM-ELEMENT-TYPE socket) '(UNSIGNED-BYTE 8))
                           ;; read the whole file in one system call
                           (EXT:READ-BYTE-SEQUENCE data socket)
                           (WITH-OPEN-FILE (out file :direction :output
                                                :ELEMENT-TYPE '(UNSIGNED-BYTE 8))
                             ;; write the whole file in one system call
                             (EXT:WRITE-BYTE-SEQUENCE data out))
                           data))))))
(DEFUN wput (host page file &OPTIONAL (port 80))
  (WITH-OPEN-STREAM (socket (SOCKET:SOCKET-CONNECT port host :EXTERNAL-FORMAT :DOS))
    (WITH-OPEN-FILE (in file :direction :inptut :ELEMENT-TYPE '(UNSIGNED-BYTE 8))
      (LET* ((length (FILE-LENGTH in))
             (data (MAKE-ARRAY length :element-type '(UNSIGNED-BYTE 8))))
        ;; some servers may not understand the "Content-length" header
        (FORMAT socket "PUT ~A HTTP/1.0~%Content-length: ~D~2%" page length)
        (SETF (STREAM-ELEMENT-TYPE socket) '(UNSIGNED-BYTE 8))
        (EXT:READ-BYTE-SEQUENCE data in)
        (EXT:WRITE-BYTE-SEQUENCE data socket)))
    ;; not necessary if the server understands the "Content-length" header
    (SOCKET:SOCKET-STREAM-SHUTDOWN socket :output)
    ;; get the server response
    (LOOP :for line = (READ-LINE socket nil nil) :while line :collect line)))

32.4.2. Socket API Reference

(SOCKET:SOCKET-SERVER &OPTIONAL port &KEY :INTERFACE :BACKLOG)

This function creates a passive socket an binds a port to it. The server exists to watch for client connection attempts.

The optional argument is the port to use (non-negative FIXNUM, 0 means assigned by the system).

The :BACKLOG parameter defines maximum length of queue of pending connections (see listen) and defaults to 1.

The :INTERFACE parameter specifies the interface(s) on which the socket server will listen, and is either a STRING, interpreted as the interface IP address that will be bound, or a socket, from whose peer the connections will be made.

Default is (for backward compatibility) to bind to all local interfaces, but for security reasons it is advisable to bind to the loopback interface "127.0.0.1" if you need only local connections.

(SOCKET:SOCKET-SERVER-CLOSE socket-server)

Closes down the server socket. Just like streams, SOCKET:SOCKET-SERVERs are closed at garbage-collection. You should not rely on this however, because garbage-collection times are not deterministic and the port assigned to the server socket cannot be reused until it is closed.

(SOCKET:SOCKET-SERVER-HOST socket-server) (SOCKET:SOCKET-SERVER-PORT socket-server)

Returns the host mask indicating which hosts can connect to this server and the port which was bound using SOCKET:SOCKET-SERVER.

(SOCKET:SOCKET-WAIT socket-server &OPTIONAL [seconds [microseconds]])

Wait for a fixed time for a connection on the socket-server (a SOCKET:SOCKET-SERVER). Without a timeout argument, SOCKET:SOCKET-WAIT blocks indefinitely. When timeout is zero, poll. Returns T when a connection is available (i.e., SOCKET:SOCKET-ACCEPT will not block) and NIL on timeout.

(SOCKET:SOCKET-ACCEPT socket-server &KEY :ELEMENT-TYPE :EXTERNAL-FORMAT :BUFFERED :TIMEOUT)

Waits for an attempt to connect to the socket-server and creates the server-side bidirectional SOCKET:SOCKET-STREAM for the connection.

SIGNALs an ERROR if no connection is made in that time.

(SOCKET:SOCKET-CONNECT port &OPTIONAL [host] &KEY :ELEMENT-TYPE :EXTERNAL-FORMAT :BUFFERED :TIMEOUT)

Attempts to create a client-side bidirectional SOCKET:SOCKET-STREAM. Blocks until the server accepts the connection, for no more than :TIMEOUT seconds. If it is 0, returns immediately and (probably) blocks on the next i/o operation (you can use SOCKET:SOCKET-STATUS to check whether it will actually block).

(SOCKET:SOCKET-STATUS socket-stream-or-list &OPTIONAL [seconds [microseconds]])

Checks whether it is possible to read from or write to a SOCKET:SOCKET-STREAM or whether a connection is available on a SOCKET:SOCKET-SERVER without blocking.

This is similar to LISTEN, which checks only one STREAM and only for input, and SOCKET:SOCKET-WAIT, which works only with SOCKET:SOCKET-SERVERs.

We define status for a SOCKET:SOCKET-SERVER or a SOCKET:SOCKET-STREAM to be :ERROR if any i/o operation will cause an ERROR.

Additionally, for a SOCKET:SOCKET-SERVER, we define status to be T if a connection is available, i.e., is SOCKET:SOCKET-ACCEPT will not block, and NIL otherwise.

Additionally, for a SOCKET:SOCKET-STREAM, we define status in the given direction (one of :INPUT, :OUTPUT, and :IO) to be

Possible status values for various directions:

:INPUT status:

`NIL`	reading will block
`:INPUT`	some input is available
`:EOF`	the stream has reached its end

:OUTPUT status:

`NIL`	writing will block
`:OUTPUT`	output to the stream will not block

:IO status:

output status	input status
output status	`NIL`	`:INPUT`	`:EOF`
`NIL`	`NIL`	`:INPUT`	`:EOF`
`:OUTPUT`	`:OUTPUT`	`:IO`	`:APPEND`

Possible values of socket-stream-or-list:

SOCKET:SOCKET-STREAM or SOCKET:SOCKET-SERVER: Returns the appropriate status, as defined above (:IO status for SOCKET:SOCKET-STREAM)
(SOCKET:SOCKET-STREAM . direction): Return the status in the specified direction
a non-empty list of the above: Return a list of values, one for each element of the argument list (a la MAPCAR)

If you want to avoid consing[3] up a fresh list, you can make the elements of socket-stream-or-list to be (socket-stream direction . x) or (socket-server . x). Then SOCKET:SOCKET-STATUS will destructively modify its argument and replace x or NIL with the status and return the modified list. You can pass this modified list to SOCKET:SOCKET-STATUS again.

The optional arguments specify the timeout. NIL means wait forever, 0 means poll.

The second value returned is the number of objects with non-NIL status, i.e., “actionable” objects. SOCKET:SOCKET-STATUS returns either due to a timeout or when this number is positive, i.e., if the timeout was NIL and SOCKET:SOCKET-STATUS did return, then the second value is positive (this is the reason NIL is not treated as an empty LIST, but as an invalid argument).

Note that SOCKET:SOCKET-STATUS may SIGNAL a STREAM-ERROR. This happens if the SOCKET:SOCKET-STREAM receives an RST packet, see tests/econnreset.lisp.

This is the interface to select (on some platforms, poll), so it will work on any CLISP STREAM which is based on a file descriptor, e.g., EXT:*KEYBOARD-INPUT* and file/pipe/socket STREAMs, as well as on raw sockets.

(SOCKET:SOCKET-STREAM-HOST socket-stream) (SOCKET:SOCKET-STREAM-PORT socket-stream)

These two functions return information about the SOCKET:SOCKET-STREAM.

(SOCKET:SOCKET-STREAM-PEER socket-stream [do-not-resolve-p])

Given a SOCKET:SOCKET-STREAM, this function returns the name of the host on the opposite side of the connection and its port number; the server-side can use this to see who connected.

When the optional second argument is non-NIL, the hostname resolution is disabled and just the IP address is returned, without the FQDN.

The socket-stream argument can also be a raw socket.

(SOCKET:SOCKET-STREAM-LOCAL socket-stream [do-not-resolve-p])

The dual to SOCKET:SOCKET-STREAM-PEER - same information, host name and port number, but for the local host. The difference from SOCKET:SOCKET-STREAM-HOST and SOCKET:SOCKET-STREAM-PORT is that this function asks the OS (and thus returns the correct trusted values) while the other two are just accessors to the internal data structure, and basically return the arguments given to the function which created the socket-stream.

The socket-stream argument can also be a raw socket.

(SOCKET:SOCKET-STREAM-SHUTDOWN socket-stream direction)

Some protocols provide for closing the connection in one direction using shutdown. This function provides an interface to this UNIX system call. direction should be :INPUT or :OUTPUT. Note that you should still call CLOSE after you are done with your socket-stream; this is best accomplished by using WITH-OPEN-STREAM.

All SOCKET:SOCKET-STREAMs are bidirectional STREAMs (i.e., both INPUT-STREAM-P and OUTPUT-STREAM-P return T for them). SOCKET:SOCKET-STREAM-SHUTDOWN breaks this and turns its argument stream into an input STREAM (if direction is :OUTPUT) or output STREAM (if direction is :INPUT). Thus, the following important invariant is preserved: whenever

a STREAM is open (i.e., OPEN-STREAM-P returns T) and
a STREAM is an input STREAM (i.e., INPUT-STREAM-P returns T)

the STREAM can be read from (e.g., with READ-CHAR or READ-BYTE).

The socket-stream argument can also be a raw socket.

(SOCKET:SOCKET-OPTIONS socket-server &REST {option}*)

Query and, optionally, set socket options using getsockopt and setsockopt. An option is a keyword, optionally followed by the new value. When the new value is not supplied, setsockopt is not called. For each option the old (or current, if new value was not supplied) value is returned. E.g., (SOCKET:SOCKET-OPTIONS socket-server :SO-LINGER 1 :SO-RCVLOWAT) returns 2 values: NIL, the old value of the :SO-LINGER option, and 1, the current value of the :SO-RCVLOWAT option.

The socket-stream argument can also be a raw socket.

(SOCKET:STREAM-HANDLES stream)

Return the input and output OS file descriptors of the stream as multiple values. See Section 33.17, “Raw Socket Access”.

32.4.3. Argument `:TIMEOUT`

The :TIMEOUT argument specifies the number of seconds to wait for an event. The value may be a

a non-negative REAL or
a LIST (sec usec) or
a CONS cell (sec . usec).

Prev	Up	Next
32.3. The Foreign Function Call Facility	Home	32.5. Multiple Threads of Execution