After asking on twitter on how to solve this problem, @kaleidic pointed me to this guide, which tells you to modify the Java preferences to run in 32 bit mode. What I didn't liked from that approach is that it seems to modify the general preferences of Java. After some searches I found that there's an option for the java executable -d32, that makes it run in 32 bits.
To apply this option simply go to the place where you installed J and there edit bin/jwd.
In that file you will find a line like this:
java -Xss8000000 -Xdock:name=J -Xdock:icon=bin/icons/jred.icns -jar bin/j.jar "$@"
which you have to modify to:
java -d32 -Xss8000000 -Xdock:name=J -Xdock:icon=bin/icons/jred.icns -jar bin/j.jar "$@"
And that's it, then simply double click on the J icon and it should work as expected.
NOTE: The line breaks that you see in the execution code are inserted by blogger, as you'll see on the jwd file, there are no line breaks there.