Re: macros

Kaz Kylheku wrote:


Well, I've never heard of it. Though if it exists despite that, the name
is quite a remarkable coincidence, wouldn't you say?


No, the Lisp language I'm talking about in this thread is Common Lisp,
and a lot of the premises of my statements in this thread have in fact
been statements about Common Lisp made by comp.lang.lisp regulars during
the course of this thread. If you disagree with some of those premises
it is because, as has been mentioned several times before, you disagree
among yourselves. I have nothing to do with it and you should get your
own house in order before looking to cast blame elsewhere.


As with all such arguments, proves nothing about the matter under debate.


No. You use different names, different computers, different news
services, and do not agree 100% with one another. (You use slrn on Linux
and the eternal-september.org NNTP service, whereas he uses Firefox on a
Mac to post via Google Groups.)

Do at least try to think before leaping to the attack. It would save you
many embarrassing stumbles like that last one.


I just said that. Why are you extracting little pieces of what I said
and repeating them, apparently thinking that by doing so you are
successfully attacking me? For that matter, why attack me at all? I pose
no threat to you. I also cannot be made to provide money, goods of
value, or similarly by insulting me. You cannot coerce me nor can I harm
you here so neither of the logical reasons for attacking, either to
plunder resources by force or to neutralize a threat, seem to apply. It
follows that your reasons for attacking are ILlogical, most likely
emotional. You would be better served by getting therapy for your issues
than by continuing to lash out at strangers via usenet. We're not here
to be your stress-relievers nor is that going to do any long term good
anyway, though it might be momentarily cathartic for you.


Once again, you miss my point, I suspect intentionally. The type system
the compiler natively knows about is small and finite. It's a bit like
if Java had only int, double, boolean, and all those other primitive
types, and Object, and the compiler didn't know Exception from
Enumeration, just that they were both Objects. You just invoked methods
on objects and got a runtime exception if the actual run-time type
didn't have a method with that name.

Fortunately Java is not like that but in any system where the compiler
and language spec have no specific recognition of individual
user-defined classes as types all type checking (and dispatch of
methods!) has to be done at runtime.


For the umpteenth time, there is no Seamus MacRae Lisp, at least so far
as I know, and even if there is, it's irrelevant. Why do you keep
bringing it up as if it were relevant? Note that of all the people in
this thread, you are the one who keeps mentioning it where previously
only Common Lisp had been under discussion. You keep bringing the
subject up, not I.

I think it is a red herring you've dredged up out of the deeps in order
to muddy the waters.


Because they're different. I could create a Java class, FooObject, that
had a String field "type" and a name in this, and its own internal
dispatch mechanism based on the name, so you'd call
someFooObject.do(methName) and depending on methName and
someFooObject.type a different Runnable (say) would get invoked.

This would be a first step towards creating a user-added object system
inside of Java. You wouldn't want to do it, because it would be slow and
inefficient and Java gives you a perfectly decent object system for free
that is much more efficient. But you could do this.

And guess what: it would be completely invisible to javac! So the
programmed-in notion type in the "type" field of the FooObject class is
not compile-time. Even when the "type" field came from a string literal
in some code somewhere, and so was a constant, it would still not be
something javac knew anything about, save that it was a field of that
class of type String. It would be up to program code to reason about the
intended semantics of that field; javac would be agnostic of these
semantics.

Similarly, CLOS is built on top of Common Lisp in the same manner as
FooObject with its explicit type field and its explicit dispatch system
coded in the host language. CLOS classes are a "programmed-in notion"
type system. Oh, I'm sure they've tweaked things to make sure it's much
more efficient than FooObject likely ever could be, using macros to do
as much as possible at compile time for example. And as someone has
mentioned, some CL implementations have a compiler that optimizes
certain kinds of CLOS use. But a conforming CL implementation can't
change the semantics of CLOS or any other code -- optimizations that
affect only performance are one thing, but actually changing behavior is
another. So everything that CL's standard says must compile, must
compile; everything it says must behave a certain way at runtime, must
behave that way at runtime. The implementation therefore must pretend it
doesn't know beans about CLOS, except for performance optimizations,
even if it actually does (so that it can do those optimizations).

Not only that, implementations are widespread that genuinely don't know
anything about CLOS.

The compiler therefore must be agnostic about types other than the
primitive type system I mentioned, the one analogous to
Java's-primitive-types-plus-Object. Except for the optimizer, which in
SOME cases may know a BIT more.

Keep in mind, also, that anyone can bolt on their own non-CLOS object
system that might look similar to CLOS, including constructing its
objects in almost the same way and storing classes and class information
in almost the same way. Any such system that is legal CL code must
compile and run flawlessly in all conforming implementations, which
means not only can they not do anything BUT tweak the speed of CLOS
code, but even these optimizations will be of strictly limited scope.
They cannot, for instance, outright break code that triggers them but
isn't really CLOS; pessimize it by making assumptions true of CLOS but
false of the custom object system, maybe, but not outright break it. In
particular the compiler cannot produce diagnostics for code that appears
to be bad or questionable CLOS code because it might be legitimate
non-CLOS code masquerading as bad CLOS code, and if so it has to work as
it would if compiled with a no-frills basic CL compiler. So the
optimizations that can be performed amount to a) laying out data in
memory and possibly emulating some data structures using different data
structures, while preserving semantics; and b) optimizing for the
common-in-CLOS case in various ways. Both might pessimize quasi-CLOS
systems, and neither should actually break anything.

(With respect to data layout, some Lisps fake constructed-in-one-go
lists that are likely to be immutable by using arrays instead of linked
lists in memory. Doing so pessimizes things if the list is subsequently
subjected to deletion other than at its head or most other modifying
operations other than changing the car values in the list. It does not
however make the semantics wrong and optimizes for the common case for
these types of list. Yes, I do know more about Lisp than you gave me
credit for. Perhaps it's time you gave up?)


Actually, like any Lisp function it takes any argument types. It *hopes*
they are types, and presumably generates some sort of exception at
runtime if one of them is not...


This notion of "known at compile time" is meaningless. If

someFooObject.type = "scuzz"

is being compiled, the "scuzz" type is "known at compile time" to javac,
since it is "named right there", but javac doesn't know it from any
other string literal. Similarly a conforming CL implementation can't
safely make assumptions about the purpose of a symbol: maybe the coder
intends it to name a type in some high-up layer of program design but
maybe they do not. Assuming it's a type and enforcing anything based on
those assumptions, or making certain kinds of decisions based on those
assumptions, will break conforming code, making the implementation
NONconforming. The most it can do is *guess* that the symbol names a
type of some sort and then do something that will make the code run
faster if the guess proves to be right but that will not alter its
behavior (speed and memory-use aside) even in the case that the guess
proves to be wrong. It can, for instance, organize the code such that if
the symbol IS a type name, the CPU's branch prediction will speed things
up a lot (method dispatch perhaps?), but the code will still work even
if it's not. Cache misses is another area. If the guess suggests a
particular branch of an if will execute much more often than the other,
it can make the machine code do an "if (predicate) goto far-away;
probably common case; RET; far-away: probably rare case" and this might
reduce cache misses versus the other way around, even if it means
negating the predicate used in the input code and exchanging the two
branches. If the guess is wrong, maybe the result is more cache misses
instead of fewer, or no change, but the code still works.


Surely it cannot. What if subtypep is not what it expected because
someone wrote their own version instead of using CLOS? After all it's
perfectly legal CL, just not legal CLOS. One of you provided an example
earlier of replacing car and cdr with content-of-address-register and
content-of-decrement-register, which changed aspects of the language.
Someone could likewise change subtypep to subtypeq (preferring "query"
to "predicate" perhaps) or make subtypep do something else.


No, CLOS has these types in that subtype relationship. Lisp itself has a
bunch of cons cells and symbols and a function that might, in theory,
mean anything, even if they do in this case happen to mean that.


See above, or grep post for "FooObject". Just a sketch of the start of
an implementation mind you, without any code even, just description.


Your side is the one making the extraordinary claim, and therefore the
one required to furnish extraordinary evidence. The burden of proof is
on the party making the extravagant, unusual, counterintuitive,
counter-commonsense, or counter-many-peoples'-experience claim and not
on the party making the conventional claim.


This statement is true but irrelevant. We are discussing type; what ELSE
can be made part of the static, compiler-groks-it set of stuff is not
important now. It's not, after all, like they tend to be mutually
exclusive, so having the compiler enforce type safety means not being
able to have it do or check something else.


Is this a trick question?

It's meaningless, because the formulation of the question presupposes
something that is false, namely, that I am obsessed with anything at all.

I need no rational justification for being something that I am not.

Moreover, I need supply you with no rational justification for anything
whatsoever. You have no authority over me, and therefore I need not ever
justify anything to you. That you apparently believe otherwise is
disturbing.

Furthermore, we are discussing Java vs. Lisp. Trying to change the
subject from that to me (e.g., asking what my justification is for X) is
illegitimate, perhaps an intentional attempt to cloud the issue and
certainly not part of any rational argument on the topic originally at
hand. It matters not a whit to the truth or falsity of any statement
about Java or Lisp what my reasons are for thinking anything, if that
statement is objective; and if it is subjective, then there can be no
fruitful arguing over its truth-value anyway!


Yes, because type safety is present in most programming languages.

Except that Lisp is not one of that set, and so type errors are not
"among the easiest to debug" in Lisp.

Nice try, moving from specific to general when the specific case is
exceptional in some way. Too bad I'm hip to your rhetorical bag of
tricks and know how to counter each and every one of 'em.

For those who really, really still fall for that old gag and need a
specific disproof of what this guy implied, just note that if a program
creates a list intended to be of integers, drops a Frog in there, waits
quite a long time doing other things, and then somewhere else in the
program some other code receives that list and tries to tot up the
numbers for whatever reason, the explosion will happen quite far from
the place and time of the error. The stack trace will point to nowhere
near where the type error actually occurred, and the poor maintenance
programmer will have to resort to a) sprinkling debug prints liberally
around the site of the crash, b) once it's localized to that particular
list getting noninteger objects in it, tracing the data flow to find all
the places it gets list references from; c) in all code that inserts
into any list that might subsequently reach that method, add debug
prints to catch any addition of a noninteger to the list and make the
program dump core then&there; and then d) run it and stress it until the
crash reproduces again, hopefully this time pointing to the erroneous
insertion. And then he has to find out how the Frog got *there*!

I've done this. Java still lacks compile-time type-safety for
nulls-not-getting-where-they-don't-belong. With generics Java finally
catches it as soon as you try to put a Frog into a List<Integer> before
even running the program but it still can't catch you putting a null in.
So I've had to track down where spurious nulls snuck into things using
exactly the kind of debugging described, and believe me, it's a LOT more
work than seeing a red squiggly underline in NetBeans and immediately
knowing a) that there's a bug and b) what and where it is.

Untyped collections and holders are particularly bad because of the time
and space shifting they enable between initial type error and symptoms
emerging. A wrong-type object (or a null) in a collection can be a
ticking bomb that goes unnoticed for weeks. The collection might even be
schlepped over the network or persisted across sessions in between error
occurring and subsequent detonation.


Another rhetorical trick, this time some kind of strawman argument. Who
said anything about "simplified and"?


Read almost any post by me to this thread. Or just read the above, where
I restated one strong argument supporting the case, with eyewitness
testimony for evidence (it doesn't get much better than that; juries
LOVE eyewitness testimony, sometimes even more than they love forensics).


Obviously, if it did not, it would not have been possible for me to have
been 1/3 of the way through it. Therefore your statement conveys no
information that I did not already know, and thus it is useless.

Perhaps if you avoided placing useless baggage like this, and like the
various snide asides, ad hominem quips, and pointless personal questions
(e.g. "how do you justify thinking xyz?", then your posts would be of a
more tractable length.


Oh, you'll find that I'm just full of surprises. I type well in excess
of 500cpm on a good day, among other things.


Only because several people ganged up on me for some inexplicable
reason. And I'm not quite single-handed. Though we do not know each
other and are not coordinating our efforts, it seems Series Expansion is
on my side rather than yours. For a time Lew was as well, before getting
bitten by that raccoon or whatever it was and developing that
unfortunate case of lyssavirus.


I can think of three cases where this seems particularly likely to occur.

1. Systems with a lot of type inference and implicit typing, like
    ML-family languages. Java has explicit types on everything.
2. Systems lacking type parametrization. Collections suffer from
    the ticking-bomb problem in C and older Javas. Java 5 and 6
    and C++ provide tools to type collections and avoid this]
    problem.
3. Cases where the type system was intentionally subverted. Using
    variables of type Object and lots of last-minute casting,
    instanecof tests, or explicit type fields in Java, for instance.


Wikipedia: http://en.wikipedia.org/wiki/Type_system

"Advocates ... have suggested that almost all bugs can be considered
type errors, if the types used in a program are properly declared by the
programmer or correctly inferred by the compiler.[2]"

Note the existence of a citation. It's to a paper the link to which is
broken, but which CiteSeer has indexed, so it's real. The claim there is
essentially 100%. I thought that outlandish enough to reduce it by more
than a factor of three, more generous of me than had I split the
difference and said 50%.

Furthermore, Pierce has a nice description of the subjective effects of
static type checks in Types and Programming Languages: "In practice,
static typechecking exposes a surprisingly broad range of errors.
Programmers working in richly typed languages often remark that their
programs tend to "just work" once they pass the typechecker, much more
often than they feel they have a right to expect. One possible
explanation for this is that not only trivial mental slips ... but also
deeper conceptual errors ... will often manifest as inconsistencies at
the level of types."


It is when an object is used where it cannot possibly work because it is
semantically the wrong kind of object. I thought everyone here woukd
know this.


Of course, but the compiler wouldn't detect it. It could only ever
manifest as a run time failure of some sort.


Of course not. However, that's comparing apples and oranges. The proper
apples-to-apples comparison is to develop two programs with the same
requirements, one using static typing and one eschewing it.


Type names can be autocompleted pretty intelligently by any decent IDE.

Time spent programming is dominated by thinking, not typing, anyway.


That's one reason why we have OO and polymorphism. If we need more than
one type of Frob we can subclass it. If we need bignums we can generify
to Number or just search and replace* on some type names and operators,
test, and fix anything that got wonky.

* Being smart enough about it to not use bignum loop indices or other
such stupidity. The compiler will catch it if we fuck up and change an
array index.

More usually, requirement changes mean we still have a Person type but
it holds some new or different data. This means changes to the class,
and some changes to client code of that class, but not typically a
drastic overhaul of the type system.


No. You should know me better than that by now. Evidence and experience,
of course, and I also provided a few citations for you to chew on, above.


I already did, but I have yet to see any from you.


I'm doing nothing of the sort and I have experience on both sides of the
fence, though with more Smalltalk than Lisp experience on the dynamic side.

You shouldn't make assumptions about me and then attack those
assumptions. When they turn out to be wrong, and they will, you will end
up looking foolish.


At least it CAN. Make "some hidden variable" that has no name, that is.
If the programmer had to implement "synchronized" they'd have to create
an explicit variable, which would have to have a name, which would then
be able to collide, or would be global.

Certainly multiple evaluation was not an option even regardless of side
effects. The expression's value might change and then it might raise one
semaphore and then lower another, breaking things. (Consider, for example:

while (node != null) {
     Node x = anotherNode();
     synchronized (node) {
         // do things with node and x
         node = node.next;
     }
}

Make a macro of this that just goes "{lock(node); body unlock(node);}"
and you get this:

{
     lock(node);
     // do things with node and x
     node = node.next;
     unlock(node);
}

Oops: locked one node and unlocked another.

OK, lets try "{Node x = node; lock(x); body unlock(x);}":

{
     Node x = node;
     lock(x);
     // do things with node and x
     node = node.next;
     unlock(x);
}

Oops: Now we lock and unlock the same object, and it's the right
opbject, but the // do things with node and x code now uses the wrong x.
Nice job!

Of course "Node x = node; {lock(x); body unlock(x);}" is even worse as
it clobbers the value returned by anotherNode() AND has the above
problem. Then there's "static Node x = node; {lock(x); body unlock(x);}"
(pretending we can make a local static variable like in C). Now it's not
reentrant (x will get clobbered if // do things with node and x ends up
invoking the same macro again, which it will if if synchronizes on
ANYTHING ELSE, and nested synchronized blocks are very common!!) and
it's not threadsafe. Let's make it threadsafe at least, shall we?

"static Node x; static final Object lock = new Object(); synchronized
(lock) { lock(x); body unlock(x);}"

....

uh-oh.

OK, time to give up. Java's "synchronized" cannot be made a user-defined
macro without some kind of catastrophic failure or another happening.
Every attempt to fix one problem causes another. It's like a bump in the
rug that pops up somewhere else whenever you try to smooth it away. I
think this is what Series was getting at before he disappeared from the
face of the planet, too.

So no, synchronized isn't quite like a predefined macro, because it can
use nameless variables, a luxury user-written code in the language
cannot have, for once you've made an anonymous reference how do you use
it again? You can't. A literal C-preprocessor-style macro that just
replaces its calls with itself and the formal parameters in the result
with the actual parameters certainly can't, because the variable name
"x" would be hard-coded and could always collide with something. But
even a Lisp-style macro can't, which can compute its replacement. In

while (node != null) {
     Node x = anotherNode();
     synchronized (node) {
         // do things with node and x
         node = node.next;
     }
}

no function of the name "node" and the code snippet
"// do things with node and x
node = node.next"

can reliably compute a name that will not cause a problem at that site.
OK, it could avoid using "x" after seeing "x" in the // do things with
node and x part, and at first it looks like you've won (but only if a
lot of very complex code is added to the macro just to compute a safe
name at each call site), until you realize that the need to be able to
nest synchronized blocks means the need to be able to nest macros.

Uh-oh.

Because now // do things with node and x can be a macro call that does
not have a literal "x" in it, but produces "x" by some more indirect
method, and determining whether it will ever produce an expression
containing "x" as a variable name or not is going to be equivalent to
the halting problem.

You're sunk.

The key advantage javac has over your precious Lisp macro is that javac
can compute its "x" from the entire code for the method, not just the
expression in parentheses and the body of the synchronized block. That,
and it needn't worry about macros computing variable names on the fly in
ways it can't predict. Even ignoring the fact that it also doesn't have
to. Lisp macros necessarily can nest and necessarily have tunnel vision.


Unbelievable. Literally unbelievable. I had to pinch myself to make sure
I was not dreaming.

Is someone really arguing with me, on the topic of computer science,
that shrugs and says "what's the difference" when someone says "C, not
assembly"?

I think we're done here.


Yep!


Calling me names does not an argument in favor of Lisp make.

In fact, you're providing circumstantial evidence in favor of Java, by
way of "Lisp programmers are a rude and surly lot, long on
trash-talking, short on logic skills, and prone to magnify every minor
disagreement of opinion into World War III. Who'd EVER want to have to
work alongside some, or risk becoming that way themselves?" The first
sentence there may not be true, but it is not an unreasonable hypothesis
based on what I have observed here recently, and every time you throw
another insult at me, you furnish one more data point that helps make a
statistical case for its being true.


Of course not, since Lisp coding doesn't tend to involve Bison.


Seamus MacRae lisp, if it even exists, is irrelevant here. I was talking
about Common Lisp. Please do not keep trying to change the subject.


One more data point.


Who said they were? We were talking about a (presumably implemented in
Lisp) pre-processor that altered the code, remember? One that one of you
mentioned a week or so ago.


Tunnel vision can be a sign of stroke, certain poisonings, and incipient
heart failure. Seek medical help.


I didn't know 3 + 3 = 17. And I still don't.


Don't believe every picture you see on a Web site. Especially if it
shows something implausible like a kangaroo with wings dive-bombing a
tennis court, the phrase "all your base" spelled out by craters on the
moon, or embedded graphics in an application running in an xterm.


Someone who makes such obviously false statements as "the only Emacs is
Seamus MacRae Imaginary Emacs". Let's see, there's the original, there's
GOSMACS, there's ... oh, hell, do we even need more than one, let alone
two, let alone still more counterexamples to prove your statement false?
And I'm sure you've heard of at least one of those counterexamples
before which means you weren't merely mistaken, you were, indeed, lying.

And since when is "an application running in an xterm cannot display
true graphics" a "pre-conceived notion" rather than "bleeding obvious"?


There are far too many of them.


I deal with very few on a regular basis.


Why should I? I know it's false, since, for instance, I'm not one.


Common Lisp!! What else have we been discussing for what feels like
weeks now?

I think you're feeling additional symptoms of your incipient stroke.
Dial 9-1-1 before it's too late.