Why does sun.misc.Unsafe exist, and how can it be used in the real world?

Question

I came across the sun.misc.Unsafe package the other day and was amazed at what it could do.

Of course, the class is undocumented, but I was wondering if there was ever a good reason to use it. What scenarios might arise where you would need to use it? How might it be used in a real-world scenario?

Furthermore, if you do need it, does that not indicate that something is probably wrong with your design?

Why does Java even include this class?

Solution 1

examples

1. VM "intrinsification." ie CAS (Compare-And-Swap) used in Lock-Free Hash Tables eg:sun.misc.Unsafe.compareAndSwapInt it can make real JNI calls into native code that contains special instructions for CAS

   read more about CAS here http://en.wikipedia.org/wiki/Compare-and-swap

2. The sun.misc.Unsafe functionality of the host VM can be used to allocate uninitialized objects and then interpret the constructor invocation as any other method call.

3. One can track the data from the native address.It is possible to retrieve an object’s memory address using the java.lang.Unsafe class, and operate on its fields directly via unsafe get/put methods!

4. Compile time optimizations for JVM. HIgh performance VM using "magic", requiring low-level operations. eg: http://en.wikipedia.org/wiki/Jikes_RVM

5. Allocating memory, sun.misc.Unsafe.allocateMemory eg:- DirectByteBuffer constructor internally calls it when ByteBuffer.allocateDirect is invoked

6. Tracing the call stack and replaying with values instantiated by sun.misc.Unsafe, useful for instrumentation

7. sun.misc.Unsafe.arrayBaseOffset and arrayIndexScale can be used to develop arraylets,a technique for efficiently breaking up large arrays into smaller objects to limit the real-time cost of scan, update or move operations on large objects

8. http://robaustin.wikidot.com/how-to-write-to-direct-memory-locations-in-java

more on references here - http://bytescrolls.blogspot.com/2011/04/interesting-uses-of-sunmiscunsafe.html

Solution 2

Just from running a search in some code search engine I get the following examples:

• Java Object Notation - use it for more efficient array processing, quoting the javadoc

Simple class to obtain access to the {@link Unsafe} object. {@link Unsafe} * is required to allow efficient CAS operations on arrays. Note that the versions in {@link java.util.concurrent.atomic}, such as {@link java.util.concurrent.atomic.AtomicLongArray}, require extra memory ordering guarantees which are generally not needed in these algorithms and are also expensive on most processors.

• SoyLatte - java 6 for osx javadoc excerpt

/** Base class for sun.misc.Unsafe-based FieldAccessors for static fields. The observation is that there are only nine types of fields from the standpoint of reflection code: the eight primitive types and Object. Using class Unsafe instead of generated bytecodes saves memory and loading time for the dynamically-generated FieldAccessors. */

• SpikeSource

/* FinalFields that are sent across the wire .. how to unmarshall and recreate the object on the receiving side? We don't want to invoke the constructor since it would establish values for final fields. We have to recreate the final field exactly like it was on the sender side. The sun.misc.Unsafe does this for us. */

There are many other examples, just follow the above link...

Solution 3

Interesting, I'd never even heard of this class (which is probably a good thing, really).

One thing that jumps to mind is using Unsafe#setMemory to zeroize buffers that contained sensitive information at one point (passwords, keys, ...). You could even do this to fields of "immutable" objects (then again I suppose plain old reflection might do the trick here too). I'm no security expert though so take this with a grain of salt.

Solution 4

Based on a very brief analysis of the Java 1.6.12 library using eclipse for reference tracing, it seems as though every useful functionality of Unsafe is exposed in useful ways.

CAS operations are exposed through the Atomic* classes. Memory manipulations functions are exposed through DirectByteBuffer Sync instructions (park,unpark) are exposed through the AbstractQueuedSynchronizer which in turn is used by Lock implementations.

Solution 5

Unsafe.throwException - allows to throw checked exception without declaring them.

This is useful in some cases where you deal with reflection or AOP.

Assume you Build a generic proxy for a user defined Interface. And the user can specify which exception is thrown by the implmentation in a special case just by declaring the exception in the interface. Then this is the only way I know, to rise a checked exception in the Dynamic Implementation of the Interface.

import org.junit.Test;
/** need to allow forbidden references! */ import sun.misc.Unsafe;

/**
 * Demonstrate how to throw an undeclared checked exception.
 * This is a hack, because it uses the forbidden Class {@link sun.misc.Unsafe}.
 */
public class ExceptionTest {

    /**
     * A checked exception.
     */
    public static class MyException extends Exception {
        private static final long serialVersionUID = 5960664994726581924L;
    }

    /**
     * Throw the Exception.
     */
    @SuppressWarnings("restriction")
    public static void throwUndeclared() {
        getUnsafe().throwException(new MyException());
    }

    /**
     * Return an instance of {@link sun.misc.Unsafe}.
     * @return THE instance
     */
    @SuppressWarnings("restriction")
    private static Unsafe getUnsafe() {
        try {

            Field singleoneInstanceField = Unsafe.class.getDeclaredField("theUnsafe");
            singleoneInstanceField.setAccessible(true);
            return (Unsafe) singleoneInstanceField.get(null);

        } catch (IllegalArgumentException e) {
            throw createExceptionForObtainingUnsafe(e);
        } catch (SecurityException e) {
            throw createExceptionForObtainingUnsafe(e);
        } catch (NoSuchFieldException e) {
            throw createExceptionForObtainingUnsafe(e);
        } catch (IllegalAccessException e) {
            throw createExceptionForObtainingUnsafe(e);
        }
    }

    private static RuntimeException createExceptionForObtainingUnsafe(final Throwable cause) {
        return new RuntimeException("error while obtaining sun.misc.Unsafe", cause);
    }


    /**
     * scenario: test that an CheckedException {@link MyException} can be thrown
     * from an method that not declare it.
     */
    @Test(expected = MyException.class)
    public void testUnsingUnsaveToThrowCheckedException() {
        throwUndeclared();
    }
}

View more solutions

Comments

[Andy Lynch]

The JDK devs are currently reviewing this API for possible transformation into a public API in Java 9. If you use it, it's worth taking 5 minutes to fill out the survey: surveymonkey.com/s/sun-misc-Unsafe .

[Jon Clements]

This post is being discussed on meta: meta.stackoverflow.com/questions/299139/…

[skinnypinny]

but according to the discussion on the thread below, uncontented volatile are almost as fast as non-volatiles anyway stackoverflow.com/questions/5573782/…

[Tim Bender]

I'd never even heard of this class ... I've told you about it so many times! sigh + :(

[skinnypinny]

if you get the address of a field using Unsafe, it can always be changed by the GC, so isnt that operation pretty useless?

[Daniel Cassidy]

There wouldn't be any point, since Java uses a copying generational garbage collector and your sensitive information will quite probably already be located somewhere else in 'free' memory waiting to be overwritten.

[aroth]

Never heard of it either, but I love their park() documentation: "Block current thread, returning when a balancing unpark occurs, or a balancing unpark has already occurred, or the thread is interrupted, or, if not absolute and time is not zero, the given time nanoseconds have elapsed, or if absolute, the given deadline in milliseconds since Epoch has passed, or spuriously (i.e., returning for no 'reason')". Almost as good as "memory is freed when the program exits, or, at random intervals, whichever comes first".

[JimProgrammer]

get the address for the ones you have allocated

[skinnypinny]

what exactly do you mean by the one I have allocated. this seems to be used in places where objects were created using the 'new' operator, thus my question.

[JimProgrammer]

unsafe.allocateMemory and put the value

[Mike Daniels]

@Daniel, interesting, I hadn't considered that. Now you can see why I'm not a security expert. :)

[bestsss]

you can do the same w/ Thread.stop(Throwable) no need for unsafe, in the same thread you can throw anything anyways (there is no compile check)

[bestsss]

publicly available as java.util.concurrent.locks.LockSupport

[bestsss]

sun.misc.Unsafe.arrayBaseOffset and arrayIndexScale cannot be used for arraylets, besides eliminating the bounds checking. Access outside the arrays reserved array may result in memory corruption and/or seg. fault.

[amara]

actually, the SecurityManager disallows access to it only if reflection is disabled

[templatetypedef]

@sparkleshy- Can you elaborate on this?

[amara]

while getting an instance from getUnsafe does have rather strict requirements, Unsafe.class.getDeclaredField("theUnsafe") with .setAccessible(true) and then .get(null) will get it too

[templatetypedef]

@sparkleshy- I'm surprised that works - the security manager should be flagging that.

[Antimony]

You can do this purely through bytecode (Or use Lomboc to do it for you)

[Miguel Gamboa]

Regarding the point 2, I would like to know how can you invoke the constructor as any other method call? Because I did not find any way of doing that unless in bytecodes.

[Nitsan Wakart]

you cannot replace volatile semantics with plain writes and volatile reads... this is a recipe for disaster as it may work in one settings but not another. If you are looking to have volatile semantics with a single writer thread you can use AtomicReference.lazySet on the writing thread and get() on the readers(see this post for a discussion on the topic). Volatile reads are relatively cheap, but not free, see here .

[Matt Wonlaw]

"...you could use the putObjectVolatile when writing it..." I wasn't suggesting plain writes.

[gparyani]

@bestsss That method has been stubbed out and throws an UnsupportedOperationException in the current thread as of Java 8. However, the no-argument version that throws ThreadDeath still works.

[bestsss]

@damryfbfnetsi, I have not followed core jdk discussions for quite some time and no plans to move to java 8. Yet, this is quite a puzzling idea since it's trivial to be implemented by bytecode generation anyways, unless now the verifier actually checks if they method declares the throwables... but that might be backward incompatible as metadata about thrown exception was free to be discarded.

[bestsss]

The AtomicXXXUpdaters are too slow and when you really need them: CAS - you can't afford to use them actually. If you are going to do the metal you won't be using the abstraction levels and numerous of checks. Failing the CAS is a bad in a loop esp. when the hardware decides to mispredict the branch (due to high contention) but having few more compare/branches just hurts. Park/Unpark are exposed through LockSupport not AQS (that latter is more of a lock impl. than park/unpark)

[bestsss]

Doh, arraycopy uses SSE loops on x86-64 which are better than getLong/putLong (and you have to calculate the address as well)

[StaxMan]

Have you actually measured this? For shorter blocks I see consistently better performance on x86-64 when using combination of getLong/putLong: ideally I would prefer System.arraycopy() for simplicity and all; but actual testing showed otherwise for cases I have tested.

[bestsss]

yes using unsafe I could not any meaningful performance out of deflate impl. For several bytes long copies over large arrays get/putLong might work indeed when the compiler has to check lengths. Some impl. add memory fence past System.arrayCopy (can be disabled/enabled though), so that could be the real culprit.

[StaxMan]

Ok. It is possible that newer JDKs have changed this; originally when I did observe faster operation (with JDK 1.6) I was surprised too. Or perhaps I am forgetting some specific difference in usage. These are tricky (and possibly unstable) optimizations, even when they do work, and it is essential to measure effects.