Force refresh of collection JPA entityManager

Question

I am using SEAM with JPA (implemented as a Seam Managed Persistance Context), in my backing bean I load a collection of entities (ArrayList) into the backing bean.

If a different user modifies one of the entities in a different session I want these changes to be propagated to the collection in my session, I have a method refreshList() and have tried the following...

@Override
public List<ItemStatus> refreshList(){
    itemList = itemStatusDAO.getCurrentStatus();
}

With the following query

@SuppressWarnings("unchecked")
@Override
public List<ItemStatus> getCurrentStatus(){
    String s = "SELECT DISTINCT iS FROM ItemStatus iS ";
    s+="ORDER BY iS.dateCreated ASC";
    Query q = this.getEntityManager().createQuery(s);
    return q.getResultList();
}

Re-executing the query, this just returns the same data I already have (I assume it is using the 1st level cache rather than hitting the database)

@Override
public List<ItemStatus> refreshList(){
    itemStatusDAO.refresh(itemList)
}

Calling entityManager.refresh(), this should refresh from the database however I get a javax.ejb.EJBTransactionRolledbackException: Entity not managed exception when I use this, normally I would use entityManager.findById(entity.getId) before calling .refresh() to ensure it is attached to the PC but as I am refreshing a collection of entities I cant do that.

It seems like quite a simple problem, I cant believe there is no way to force JPA/hibernate to bypass the cache and hit the database?!

UPDATE TEST CASE:

I am using two different browsers (1 and 2) to load the same web page, I make a modification in 1 which updates a boolean attribute in one of the ItemStatus entities, the view is refreshed for 1 to display the updated attribute, I check the database via PGAdmin and the row has been updated. I then press refresh in Browser 2 and the attribute has not been updated

I tried using the following method to merge all entities before calling .refresh, but the entities were still not updated from the database.

@Override
public void mergeCollectionIntoEntityManager(List<T> entityCollection){
    for(T entity: entityCollection){
        if(!this.getEntityManager().contains(entity)){
            this.getEntityManager().refresh(this.getEntityManager().merge(entity));
        }
    }
}

Solution 1

You're having two separate problems here. Let's take the easy one first.

---

javax.ejb.EJBTransactionRolledbackException: Entity not managed

The List of objects returned by that query is not itself an Entity, and so you can't .refresh it. In fact, that's what the exception is complaining about. You're asking the EntityManager to do something with an object that is simply not a known Entity.

If you want to .refresh a bunch of things, iterate through them and .refresh them individually.

---

Refreshing the list of ItemStatus

You're interacting with Hibernate's Session-level cache in a way that, from your question, you don't expect. From the Hibernate docs:

For objects attached to a particular Session (i.e., in the scope of a Session)... JVM identity for database identity is guaranteed by Hibernate.

The impact of this on Query.getResultList() is that you do not necessarily get back the most up to date state of the database.

The Query you run is really getting a list of entity IDs that match that query. Any IDs that are already present in the Session cache are matched up to known entities, while any IDs that are not are populated based on database state. The previously known entities are not refreshed from the database at all.

What this means is that in the case where, between two executions of the Query from within the same transaction, some data has changed in the database for a particular known entity, the second Query would not pick up that change. It would, however, pick up a brand new ItemStatus instance (unless you were using a query cache, which I assume you're not).

Long story short: With Hibernate, whenever you want to, within a single transaction, load an entity and then pick up additional changes to that entity from the database, you must explicitly .refresh(entity).

How you want to deal with this depends a bit on your use case. Two options I can think of off the bat:

1. Have are to have the DAO tied to the lifespan of the transaction, and lazily initialize the List<ItemStatus>. Subsequent calls to DAO.refreshList iterate through the List and .refresh(status). If you also need newly added entities, you should run the Query and also refresh the known ItemStatus objects.
2. Start a new transaction. Sounds like from your chat with @Perception though that that is not an option.

---

Some additional notes

There was discussion about using query hints. Here's why they didn't work:

org.hibernate.cacheable=false This would only be relevant if you were using a query cache, which is only recommended in very particular circumstances. Even if you were using it though, it wouldn't affect your situation because the query cache contains object IDs, not data.

org.hibernate.cacheMode=REFRESH This is a directive to Hibernate's second-level cache. If the second-level cache were turned on, AND you were issuing the two queries from different transactions, then you would have gotten stale data in the second query, and this directive would have resolved the problem. But if you're in the same Session in the two queries, the second level cache would only come in to play to avoid database loading for entities that are new to this Session.

Solution 2

Try marking it @Transactional

@Override
@org.jboss.seam.annotations.Transactional
public void refreshList(){
    itemList = em.createQuery("...").getResultList();
}

Solution 3

You have to refersh entity which was returned by entityManager.merge() method, something like:

@Override
public void refreshCollection(List<T> entityCollection){
    for(T entity: entityCollection){
        if(!this.getEntityManager().contains(entity)){
            this.getEntityManager().refresh(this.getEntityManager().merge(entity));
        }
    }
}

This way you should get rid of javax.ejb.EJBTransactionRolledbackException: Entity not managed exception.

UPDATE

Maybe it's safer to return new collection:

public List<T> refreshCollection(List<T> entityCollection)
    {
        List<T> result = new ArrayList<T>();
        if (entityCollection != null && !entityCollection.isEmpty()) {
            getEntityManager().getEntityManagerFactory().getCache().evict(entityCollection.get(0).getClass());
            T mergedEntity;
            for (T entity : entityCollection) {
                mergedEntity = entityManager.merge(entity);
                getEntityManager().refresh(mergedEntity);
                result.add(mergedEntity);
            }
        }
        return result;
    }

or you can be more effective if you can access entity IDs like this:

public List<T> refreshCollection(List<T> entityCollection)
    {
        List<T> result = new ArrayList<T>();
        T mergedEntity;
        for (T entity : entityCollection) {
            getEntityManager().getEntityManagerFactory().getCache().evict(entity.getClass(), entity.getId());
            result.add(getEntityManager().find(entity.getClass(), entity.getId()));
        }
        return result;
    }

Solution 4

One of the option - bypass JPA cache for a specific query results:

    // force refresh results and not to use cache

    query.setHint("javax.persistence.cache.storeMode", "REFRESH");

many other tuning and configuration tips could be found on this site http://docs.oracle.com/javaee/6/tutorial/doc/gkjjj.html

View more solutions

Comments

[Perception]

Did you merge your objects back into the entity manager prior to calling refresh?

[DaveB]

No I didn't, the merge() method doesnt accept collections and I thought calling it on every member of the collection is a bit excessive?

[Perception]

It could be, if you have a really large list. But normal merges just synchronize the object state to the entity managers unit of work and are very fast. If merging takes too long you could always boot everything out of the second level cache with entityMgr.getEntityManagerFactory().getCache().evictAll().

[Perception]

What query are you using to retrieve your objects? Please add it to your question.

[DaveB]

Thanks, have added the query. I did try using entityManager.clear() which refreshes the list, but throws up other errors as it clears the WHOLE thing which includes data that is needed elsewhere, does .evictAll() do the same thing? I need a way to evict only the list if thats possible?

[Perception]

you can refresh a single query, add the line query.setHint("javax.persistence.cache.storeMode", "REFRESH"); right before the call to q.getResultList().

[DaveB]

Tried that and got 'javax.ejb.EJBTransactionRolledbackException: Value for hint' , is that compatible with JPA 1.0?

[Perception]

let us continue this discussion in chat

[Pace]

What isolation level are you using? If you are using REPEATABLE_READ (the default in many cases) that may be the issue.

[DaveB]

Hi @Pace, I cannot see this in any hibernate config so I assume it is the default for PostgreSQL

[Pace]

Nope, the default in PostgreSQL is READ_COMMITTED so you should be ok. For safety you could try closing the transaction and opening it back up before doing your new query.

[DaveB]

that would work but I want to keep the persistance context open (this is the whole point of the SMPC) otherwise I run into problems elsewhere in the backing bean, also that seems a bit like using a sledgehammer to crack a nut!

[sharakan]

You said that "Re-executing the query...returns the same data". Does it return the same instance of the collection, or a different collection with the same instances of the individual entities? Also, what is the version of Hibernate you are using?

[DaveB]

Thanks for the detailed answer, very useful! I now understand why the entities are not updated, but when I iterate the list and call .refresh() on each item it still doesn't give me the updated values, to answer you previous question I am using Hibernate 3.6.0, I am not sure how to check if they are new instances or not (presumably I could check the hash code for equality?).

[DaveB]

I tried this and it does prevent the exception but does not give me the updated entities from the database

[sharakan]

For the new instance check, just use ==, you want to know if the objects are the same. For .refresh, that almost certainly means that your transaction isolation settings aren't what you think they are. Can you explicitly set it to READ-COMMITED?

[sharakan]

To do this, change your Hibernate connection properties to set hibernate.connection.isolation=2

[DaveB]

Got it working, not only that but learnt a few things about hibernate in the meantime, ta!

[sharakan]

Hth. Out of curiosity what was causing that final problem with the refresh?

[DaveB]

I thought it was the isolation settings but it was actually a tweak I made to the refresh method after re-reading your answer (I needed to re-run the query as you stated in point (2) to pick up a new entity). Thanks again

[user2250246]

I tried this too, but in my case, I didn't even see the Hibernate query to refresh. Also, the entity did not reflect DB generated values too.

[woodz]

it's a valuable shorthand when not using transactions but getting the latest data from db is a must