common-lisp , Write fast Common Lisp code [closed]


Write fast Common Lisp code [closed]

Question:

Tag: common-lisp

I'm not sure, if some weird things make my Code faster:

Is it normally better to use inbuilt operations or write new specialized functions, that do the same thing? (for example a version of #'map only for vectors; my version is often faster without type declarations)

Should I define new (complicated) types to use them in declarations? (for example a typed list)

Should I define slots directly to an object? (for example px and py for a 2-dimensional object, or is it ok to use one slot pos of type vector, that I could reuse it for other purposes)


Answer:

There are a few parts to this but here is a quick braindump

PROFILE!

Use a distribution of CL that has a profiler built in, I use sbcl for example http://www.sbcl.org/1.0/manual/Statistical-Profiler.html

The nice thing about the sbcl profiler is that once you have profiled a function, if you disassemble it, the machine code is annotated with statistics. This requires some knowledge of the target machine code.

Do not underestimate your implementation: They can have advanced type and flow analysis built in and are able to, for example, pick a vector only version of map when it makes sense.

Learn compiler macros: compiler macros can shadow functions this gives you a place to put extra optimizations based on the context of the form. IT does this without replacing the function so it can still be used in a higher order way.

Learn Type declarations

I found this series of blog posts helped me understand this technique http://nklein.com/tags/optimization/page/2/ Read em all!

ONE MASSIVE NOTE: Don't ever lie to your compiler about a type. Type declarations are a way of telling your compiler you know what the type is the compiler doesn't even have to use them, and when it does it doesn't have to check you are giving it the correct thing.

Unboxed data

Some implementations are able to unbox certain datatype in certain conditions. Sorry that is vague but you will need to read up for your implementation. For sbcl the 'sbcl internals' guide is very helpful.

For example:

(make-array 100 :element-type 'single-float :initial-element 0.0)

Can be stored as a contiguous block of memory in sbcl.

PROFILE AGAIN (With realistic data)

I spent 3 hours writing a crazy compiler macro based n dimensional matrix multiplication routine and then tested it against a 1 line built in solution. For matricies below 5 dimensions there was not a big difference! For higher dimensions, yeah It rocked but that 'performance benefit' is purely academic because those code paths were never touched. Luckily I undertook the task for fun as I was asking the same question you are now.

Algorithms

All the type specifiers in the world won't give you a 100times performance increase. This comes from better techniques. Read on the maths behind the problem, implements different helper functions that have different strengths and choose between them at runtime...then go back and use compiler macros to allow lisp to choose at compile time. OR specify the technique as a higher order functions, for example make-hash-table allows you to specify the hashing function and rehash sizes, this can be crucial in getting good performance at certain sizes.

Know the limits of BigO

Algorithmic complexity means nothing if you loose all the of performance due to memory locality issues. Conversly sometime we can achieve superlinear performance characteristics if, by spliting the problem among cores, the reduced dataset now fits in the l2 cache.

BigO is a great metric but it isn't the end of the story. This is the reason assoc lists are a totally valid alternative to hash-tables for low numbers of keys and certain access profiles.

Summary

There is a golden mantra I heard from somewhere in the lisp community that works so well:

Make it Fast and then make it Fast

If nothing else follow this. Chant it to yourself!

Get the program up and running quickly, in doing so you are more likely to spot the places where you can use a better technique or algorithm to get your several-orders-of-magnitude improvement. Do use CL's own functions first. Don't trade lisp's higher order nature too early by using macros, explore how far you can go with functions.

[Edit] More notes - the following is for sbcl

So to kinda directly answer your original question:


Related:


How do I best save/read data structures?


common-lisp,clos
I want to write some data structures pointed to by FOO and BAR to a file, and to read the data structures back into the symbols FOO and BAR when I start a new session of Common Lisp. It would appear *PRINT-READABLY* allows objects to be printed in a fashion...

What is a common-lisp analogue of python's argparse?


python,common-lisp,argparse
What is a common-lisp analogue of python's argparse library for parsing command-line arguments?

Updating the window in response to CLIM frame commands


common-lisp,clim,mcclim
While trying to figure out CLIM, I ran into this example program. It's a simple maze game. The author claims to have tested it in LispWorks (and even has #+Genera in there, implying that this program would work on a real Lisp Machine), but I'm trying to get it working...

Macro that defines functions whose names are based on the macro's arguments


macros,common-lisp,symbols,sbcl
*Note: Despite having frequented StackOverflow for a long time, this is the first question that I have posted myself. Apologies if it's a bit verbose. Constructive criticism appreciated. When I define a struct in Common Lisp using defstruct, a predicate function is automatically generated that tests whether its argument is...

sbcl run-program hang when there is large amount of output from program


common-lisp,hang,sbcl
Recently I find a run-program hang issue of sbcl 1.2.7 (32bits, linux). The code as following (progn (with-open-file (s "test.out" :direction :output :if-exists :supersede) (loop repeat 900 do (write-line (make-string 76 :initial-element #\x) s))) (run-program "/bin/bash" (list "-c" "cat test.out") :output :stream)) That is when the "cat test.out" produce many...

What does backtick mean in LISP?


lisp,common-lisp,backticks
I have this macro, which rewrites define. If I remove the " ` " backtick it won't work. What is the explanation? (defmacro define ((name &rest r) body) `(defun ,name ,r ,body)) ...

Why does lisp use gensym and other languages don't?


lisp,common-lisp
Correct me if I'm wrong, but there is nothing like gensym in Java, C, C++, Python, Javascript, or any of the other languages I've used, and I've never seemed to need it. Why is it necessary in Lisp and not in other langauges? For clarification, I'm learning Common Lisp.

Dynamic 2d array in lisp


common-lisp,clisp
I want to have a 2D array in Lisp . But each row can have different number of elements(At max 5). So I thought of maintaining another single list to store the current sizes of each row, and update them whenever required. So, my code goes like this : (setq...

Need help getting CLisp to read standard input into a list


python,lisp,common-lisp,clisp
I'm working on converting some existing Python code to CLisp just as an exercise ... The program reads a list of numbers and creates mean, min, max and standard deviation from the list. I have the file-based function working: (defun get-file (filename) (with-open-file (stream filename) (loop for line = (read-line...

Common Lisp - Writing a function that detects circular lists


list,function,recursion,lisp,common-lisp
I have an assignment for my CS functional languages class where we must write a function able to detect whether or not a given list is circular at its beginning. The function has to be recursive. Let's name it circular : * (setf list1 '(a b c d)) * (rplacd...

Backquote String Interpolation


macros,lisp,common-lisp
Is it possible to use lisp's macro to do string interpolation? For instance, can I make a macro like this: (defmacro test (a) `",a") So that (test abc) returns "abc" as a string? I could probably cheat by quoting it and turning that quote into a string, but that doesn't...

Not numeric atoms LISP


lisp,common-lisp
I want to ask why this function doesn't work... (defun nenum(ls) (cond ((null ls) nil) ((listp car(ls)) (nenum (rest ls))) ((numberp car(ls)) (nenum (rest ls))) (t (cons (car ls) (nenum (rest ls)))))) Example: (nenum '(l 1 i (b) (5) s -2 p)) --> (l i s p) Thank you!...

SBCL: Gather output of run-program process while running


subprocess,common-lisp,sbcl
Dear StackExchange members, I recently began toying around with Common Lisp and want to create a web interface for administrating a modded Minecraft server. I already tried out this solution but in this case the function just hangs and never returns. My code looks like this: (defvar *proc*) (defun create-minecraft-proc...

Weird non-lisp errors with capturing labels with a lambda and `reduce`


lambda,crash,closures,common-lisp,allegro-cl
I stumbled across a very weird bug (?) while I was developing an application in Allegro Common Lisp, v9.0 for Windows. I do not get the regular lisp errors, instead I get system errors encapsulated in a lisp condition. I managed to create a simple test-case to reproduce the error...

Custom slot options don't apply any reduction to its argument


common-lisp,clos,mop
Say if I define a metaclass that enhances standard slots with a validator slot, when I pass :validator (clavier:valid-email "The email is invalid") as an option, instead of storing the result of of the expression, which is a funcallable, it stores the expression itself. Am I'm missing a step when...

Char Comparison in Common Lisp


sorting,char,comparison,lisp,common-lisp
How can i compare characters with Common Lisp? I have google it and found out that there are some functions to do the comparison, like char=, char/=, char<, char>, char<=, and char>=. But, i can't use all of the functions just like: (char= 'a 'a) (char< 'a 'b) If i...

lisp remove a the content of one list from another list


lisp,common-lisp
I have a list of string like this called F: ("hello word i'am walid" "goodbye madame") => this list contain two elements of string and I have another list call S like this ("word" "madame") => this contain two words now I want to remove the elements of the list...

Long integer to string and vice versa, operation with digits


common-lisp,sbcl
Solving the Euler project problems I get that I need to make operations with the digits of a long number normally as a string. I work in linux, emacs, slime with sbcl. For example, to get the sum of the digits of this power 2¹⁰⁰⁰, I work this way, 1)...

Common-LISP debugging (code included)


debugging,lisp,common-lisp
I hope to get a quick fix to my code, which takes in a list of numbers, numberlist, and a threshold and return the number of items in numberlist that are greater than threshold. I just can't figure out what's wrong, and I am not familiar with debugging. I am...

How to use a parameter in a macro call?


macros,lisp,common-lisp,sbcl
I have defined the following simple macro: (defmacro define-class (class-name) `(defclass ,class-name ()())) And now I want to use it in the following function: (defun create-data (mode) (define-class mode)) After compiling the last function I get the following message, the variable MODE is defined but never used. And when I...

No-argument (and) returns t


clojure,scheme,lisp,common-lisp
Both CL and Scheme define (and) to return t (or #t) with no arguments. I'm trying to understand the rationale for this. My naive assumption is that an empty set is false, and passing in zero arguments feels like passing in nothing that can be true. Edit: clojure follows the...

pathnames in Common Lisp, filenames with wildcards in them


common-lisp,filenames,wildcard,clisp,pathname
The characters * and ? are used as wildcards in pathnames. How does one refer to a filename that has ? as one of its actual characters? For example: [18]> (wild-pathname-p #p"foo") NIL [19]> (wild-pathname-p #p"foo?") T So referring to the filename "foo?" cannot be done this way. I tried...

If strings are vectors, why are they immutable?


lisp,common-lisp
if strings are vectors of characters, and a vector's elements can be accessed using elt, and elt is setf-able - then why are strings immutable?

Local dynamic binding in common lisp


closures,common-lisp,dynamic-binding
Honnestly, I'm not sure I fully understand what it means for a binding to be "dynamic" versus "lexical". But I understand that when I use defvar or defparameterto define a binding, 1. it declares a global variable 2. the binding is declared "special", so that it can be shadowed by...

Is there a way to insert raw javascript in parenscript code?


common-lisp,parenscript
The following code inserts third-party generated javascript as a string which will need to be eval'ed. (ps (let ((x (lisp (json:encode-json-alist-to-string '((:a . 1) (:b . 2)))))))) "(function () { var x = '{\"a\":1,\"b\":2}'; return null; })();" Is there a way to tell parenscript to insert a string unquoted?...

Invalid specialized parameter in method lambda list


lisp,common-lisp
I am trying to write a simple coin flip program in Common Lisp. This is the code I have (defun yn (let ht (random 1) (if (eq ht 1) (princ heads) (princ tails)) ) ) It seems simple enough, but I keep getting the error: "Invalid specialized parameter in method...

Common Lisp - Giving &rest Lists as Arguments


arguments,lisp,common-lisp,sbcl
As a result of musings around an exercism problem, I am trying to write a function that takes an input number and an arbitrary length list of divisors to test, along with the expected divisibility (i.e. remainder 0) as a boolean, returning true if all expectations are met (defaulting to...

lisp: dynamic scope vs explicit parameter passing


design-patterns,coding-style,common-lisp,anti-patterns,dynamic-scope
I see two different patterns for "output" functions in (common) lisp: (defun implicit () (format t "Life? Don't talk to me about life!")) (defun explicit (stream) (format stream "This will all end in tears.")) (defun test-im-vs-ex-plicit () (values (with-output-to-string (stream) (let ((*standard-output* stream)) (implicit))) (with-output-to-string (stream) (explicit stream)))) Is using...

Running compiled lisp program with clisp-2.49 on OS X


common-lisp,clisp
I have just started out programming in Common Lisp using GNU clisp-2.49 (compiled from source) as my implementation on OS X 10.10 Yosemite. I've written a simple "Hello World" program as follows: (EXT:SAVEINITMEM "test" :INIT-FUNCTION 'main :EXECUTABLE t) (defun main () (format t "Hello World!") (EXT:EXIT)) When I run: $...

How to Call a Go Program from Common Lisp


common-lisp
I have a Go program which cannot be rewritten in Common Lisp for efficiency reasons. How can I run it via Common Lisp? Options so far: 1. CFFI Using the foreign function interface seems to me like the "correct" way to do this. However, the research I did lead directly...

Common Lisp Object System method execution order


lisp,common-lisp,clos
I have the following two classes: (defclass person () ()) (defmethod speak ((s person) string) (format t "-A" string)) (defmethod speak :before ((s person) string) (print "Hello! ")) (defmethod speak :after ((s person) string) (print "Have a nice day!")) (defclass speaker (person) ()) (defmethod speak ((i speaker) string) (print "Bonjour!"))...

How to move a file in Lisp using rename-file


lisp,common-lisp,pathname
What's the best way to move a file in Lisp in an implementation-independent way? For example I have an image file: (setq oldpath #P"SERVER:PICTURES;TEMP;PHOTO.PNG") and I want to move it out of the TEMP directory into the PICTURES directory. This seems to work: (setq newpath (make-pathname :host (pathname-host oldpath) :directory...

What does the non-terminating-p argument of set-macro-character do?


macros,common-lisp
Set-macro-character has an optional argument called non-terminating-p. It seems to be used to indicate whether another character should be read after reading the macro character, but the reader algorithm seems to ignore this argument. Is there a difference whether I set it to true or false?

where is the text printed by C printf


io,common-lisp,cffi
I happened encounter a trouble with calling C printf function from SBCL via cffi. The problem is when I call printf function, I can't find the output text, just the return value of printf function show on the REPL. But when I quit SBCL, the output text appears on the...

Creating A Process Queue in Common Lisp


common-lisp
I have a server running Hunchentoot (CentOS and SBCL). When a user submits a specific type of post request, a subprocess is launched (run-program) which can take up to four minutes to complete. If five people perform that specific type of request at the same time, the server runs out...

Read input into string in Lisp reader macro


macros,common-lisp,reader-macro
I am trying to make a reader macro that would convert @this into "this". This is what I currently have: (defun string-reader (stream char) (declare (ignore char)) (format nil "\"~a\"" (read-line stream t nil t)) ) (set-macro-character #\@ #'string-reader ) The problem is that this requires that I put a...

How are `tagbody` and `go` implemented under the hood in Common Lisp?


compiler-construction,common-lisp
How are tagbody and go implemented in Common Lisp? Is it some form of setjmp/longjmp or is there a more elegant way of handling this? I'm writing a lispy language implemented in C and would like to have something like this....

How to use FFI:def-call-in in clisp


lisp,common-lisp,ffi,clisp
I have figured out how to make use of shared objects created from C code into Clisp using FFI:def-call-out but I am not able to figure out how to use FFI:Def-call-in. I don't know the process and actually I am confused if clisp will also create some .so file that...

Modify list and return it inside a function in lisp


lisp,elisp,common-lisp
(defun testthis (node index newvalue) (set-nth node index newvalue) node ) I would like to modify the nth element of a list in a function and then returns this list to save the modification performed. How can I do a such function in lisp?...

Get element from list of list in lisp


lisp,elisp,common-lisp
I am a beginner with lisp. I manipulate list of list: ((name1, second) (name2, second2)) The goal of my function is to get the second element of the list that have name as it first node. For example: my list is: ((name1, second1) (name2, second2)) getelement list name1 should return...

EMACS-Live + Slime error at startup


emacs,common-lisp,slime,kubuntu
Ok, I must be missing something obvious. I'm getting stuck since yesterday to launch Emacs-live + slime. I'm using EMACS 24.3.1, installed Emacs-live and it worked well (if I start emacs-live without Slime it works), downloaded Slime-Pack from git and added this line to .emacs-live.el (live-append-packs '(~/.live-packs/slime-pack/)) I'm on a...

LISP: read number from user and commpare with array index


functional-programming,lisp,common-lisp
Hello guys I'm new In functional programming Really it is not Clear for me anyone can help me ? My Question just for getting the philosophy of writing on Functional programming language for example how I can write a program in Lisp language for reading the user inputs and compare...

EVAL/APPLY: too many arguments given to F


lisp,common-lisp,clisp
Hello why do i get *** - EVAL/APPLY: too many arguments given to F on function call with nested lists parameter. I cannot figure it out, since I passed a simple nested list. (defun f (L) (cond ((NULL l) nil) ((listp (car L)) (append (F(car L))) (F(cdr L) (car (F...

SBCL: Deploying Hunchentoot application as executable


common-lisp,executable,sbcl,hunchentoot
Dear StackOverflow community, I started playing with SBCL Common Lisp and want to develop a small web application using Hunchentoot. For easy deployment I planned to save everything in a binary using sb-ext:save-lisp-and-die as I can live with the big output size. For the executable you need to supply a...

Trying to multiply all elements in a list of sublists with elements in another list in lisp


list,common-lisp
I'm trying to multiply a list with n sublists with a list with n scalars. It's supposed to work like this: (kmult-matrix '((3 4 2 4) (2 5 6 9) (1 -2 8 10)) '(2 3 5)) => ((6 8 4 8) (6 15 18 27) (5 -10 40 50))...

Understanding “let” expression in LISP


lisp,common-lisp,let
I am extremely new to lisp, had previous experience with functional programming (Haskell, SML). Why is this code returning 14, and not 10 (ie. 1 + 2y + 3 + 1)? (defvar x 1) (defun g (z) (+ x z)) (defun f (y) (+ (g 1) (let ((x (+ y...

Insertion into a list doesn't reflect outside function whereas deletion does?


list,lisp,common-lisp
I am new to Lisp. Deletion of an item in a list by a function gets reflected outside the function but insertion doesn't. How can I do the same for insertion? For example (defun test (a b) (delete 1 a) (delete 5 b) (append '(5) b) (member '5 b)) (setq...

How would I get the min/max of a list using a key


lisp,common-lisp
I currently have a list of objects, each containing a certain attribute. I would like to get the element of the list with the min attribute value. Is there a concise way of doing this? The python equivalent would be something like: min(d, key=d.get) Is there a way of getting...

Format both an expression and its result without eval


common-lisp,eval
I am trying to format an arbitrary expression, say (+ 2 3), and at the same time, its result, 5. I have the following: (defun expr-and-result (expr) (format t "~a returns ~a~%" expr (eval expr))) CL-USER> (expr-and-result '(+ 2 3)) (+ 2 3) returns 5 Though it's a simple matter...

Structuring large Lisp applications


lisp,common-lisp,quicklisp,asdf
I am currently trying to wrap my head around packages, systems & co. I now have read Packages, systems, modules, libraries - WTF? a few times, and I think I'm still having difficulties to get it right. If I simply want to split a Lisp source file into two files,...