Execute Twisted reactor.run() in a thread?

Question

I'm trying to make a Python library to use the Ubutu One API from Objective-C. Here is my source code: https://github.com/JoseExposito/U1-Finder-Plugin/blob/master/U1FinderLib/U1FinderLib.py

I need to make many calls to the API, for this reason I need to have my reactor running once instead of run it and stop it, like in the example of the Ubuntu One documentation: https://one.ubuntu.com/developer/files/store_files/syncdaemontool

Because it is not possible to run twice the reactor... And I need to execute reactor.run() in a thread because I can not block the application that uses that library!

It is possible to do that? I'm not able to run the reactor in a thread and call the Ubuntu One API synchronous.

EDIT:

I'm using this simple source code to test the idea:

#!/usr/bin/env python
import objc
import thread
import os
import time
from   twisted.internet import reactor, defer
from   ubuntuone.platform.tools import (SyncDaemonTool, is_already_running)
from   threading import Thread
NSObject = objc.lookUpClass('NSObject')

##
# Variable to get the result of the calls to the Sync Daemon.
# The result is a JSON string stored in returned_value[0].
returned_value = ['']

##
# Objective-C facade to the methods of the U1FinderLib.
class U1FinderLib(NSObject):

    def init(self):
        self = super(U1FinderLib, self).init()
        self.sync_daemon_tool = SyncDaemonTool(None)
        Thread(target=reactor.run, args=(False,)).start()
        return self

    @objc.typedSelector('@@:')
    def volumeList(self):
        print "Begin volumeList"
        reactor.callLater(0, run_command, "volume_list", [], self.sync_daemon_tool)
        print "End volumeList"
        return returned_value[0]

##
# Auxiliar functions to call to the sync daemon.
@defer.inlineCallbacks
def run_command(action, params, sync_daemon_tool):
    print "run_command"
    running = yield is_already_running()
    print "After is_already_running"
    try:
        if not running:
            returned_value[0] = '{ type:"error" reason:"Sync Daemon is not running" }'
        else:
            print "Before run_action"
            yield run_action(action, params, sync_daemon_tool)
            print "After run_action"
    except Exception, e:
        returned_value[0] = '{ type:"error" reason:"Exception: %s" }' % e

@defer.inlineCallbacks
def run_action(action, params, sync_daemon_tool):
    if action == "volume_list":
        d = sync_daemon_tool.get_folders()
        returned_value[0] = yield d.addCallback(lambda r: volume_list(r))

# Volume List
def volume_list(folders):
    volumes_json = '{ type:"volume_list" volumes: { \n\t{ volume:"' + os.path.expanduser('~/Ubuntu One') + '" subscribed:"YES" }'
    for folder in folders:
        volumes_json += ',\n\t{ volume:"' + folder['path'] + '" subscribed:"' + ('YES' if bool(folder['subscribed']) else 'NO') + '" }'
    volumes_json += '\n} }'
    return volumes_json

if __name__ == '__main__':
    py = U1FinderLib.alloc().init()
    print py.volumeList()
    print "EXIT"

And this is the output of the program:

Begin volumeList
End volumeList

EXIT

The problem is that the "run_command" function is never called

Solution

Rather than running the reactor in a thread, you should use a reactor which integrates with the application you want to use it in.

For example, perhaps you want to use the CoreFoundation reactor (since your application uses Obj-C and has "Finder" in the name).

If you really can't do that (for example, if Ubuntu One requires a different reactor - I don't know if this is the case), then you probably can run that reactor in a thread. Most reactors do support this, although if Ubuntu One requires a particular reactor, that reactor may not support threaded usage.

You didn't actually explain what problem you have when trying to run the reactor in a thread, so I can't help you understand why it that didn't work. However, it should be easy to do so. Just:

from twisted.internet import reactor
from threading import Thread

Thread(target=reactor.run, args=(False,)).start()

Keep in mind that once you choose to run the reactor in a thread, you may only use Twisted APIs in that thread.

If this doesn't work, provide more details about how it doesn't work in your question.

Comments

[user1204395]

Thank you very much for your help. I have added a sample source code in my question. If I try to use the CFReactor I have a "No module named CFNetwork" error. I must say that my application is a bundle that I inject in the Finder process to change their behavior and I can not use many features.

[user1204395]

Solved, the Core Foundation reactor was the correct solution. Thank you very much!!

[Vajk Hermecz]

I just want to stress the last sentence: "you may only use Twisted APIs in that thread". Use reactor.callLater or reactor.callWhenRunning if you want to nag things running under the reactor...

[Jean-Paul Calderone]

Neither of those APIs is thread-safe. In non-reactor threads, use reactor.callFromThread to arrange for functions to be called in the reactor thread.

[kingkupps]

Does anyone know what the False arg in this does? The twisted documentation makes no reference to it but it appears reactor.run does not complain when passing False as an argument.

[Jean-Paul Calderone]

It's the value for installSignalHandlers. Python doesn't let you install signal handlers except in the main thread.

[Vincent Wen]

Thank you @Jean-PaulCalderone , through you have a explanation here but I still google the documentation, just post it here: twistedmatrix.com/trac/wiki/…