How well different Common Lisps run FSet

I just did some quick-and-dirty benchmarking, using FSet's test suite.  It was not designed as a benchmark, but I think it still gives a useful rough indication of how well FSet runs on different platforms.

These tests were all run on an Intel Xeon "Ivy Bridge" except the first one, which was on an Apple M2 MacBook Pro.  Values are the time to run 100 iterations of the test suite; smaller is better.

  SBCL 2.5.3 on M2           1.9

  SBCL 2.5.3                 2.8

  LispWorks 8.0.1            5.8  (Personal Edition, but I doubt that matters)

  CCL 1.13                  13.3 

  Franz Allegro 11          15.7

  ECL 24.5.10               58.1

  CLASP 2.5.0               79.4

  ABCL 1.9.2                85.3

 

Yikes!  Ignoring the M2 number, that's a factor of 30 — a very wide range.  I don't think the test is entirely fair, because I develop on SBCL and haven't put any effort into optimizing for other platforms.  I suspect the CCL and Allegro times could be improved somewhat.  The poor performance of ECL and CLASP surprises me; FSet spends most of its time doing ordinary struct and simple-vector accesses, which I would think would translate well into C.  Maybe they're still doing a lot of type- and bounds-checking, even though I've requested safety 0?

As for ABCL, I think it's a remarkable achievement that it is compatible enough to run FSet at all; I can't fault it for not being a speed demon.  My guess is that the biggest gains to be had here would be from improving ABCL itself, rather than tweaking FSet.