Issue with Tribler integration in C code

Issue with Tribler integration in C code

Hi people,

Let me first introduce myself. My name is Alexandre Franke and I work (indirectly) for AHT international (mister Ron Van Herk's company) on Tribler integration into our set-top box middleware, which is written in C.

I'm currently trying to write a C version of the "Hello world!" tutorial, calling Tribler via the Python/C API. I was able to write examples 1 and 2 with no particular issue but now I'm stuck with example 3, status monitoring. What I'd like to do is to write a C callback function, then pass it to my Python Download object.

Here's a little code exerpt:

//First the code of my callback function

PyObject? * download_state_callback(PyObject? *ds) {

printf("\nDownload status: %s\n", PyString?_AsString(PyObject?_CallMethod(ds, "status", NULL)));

printf("\nDownload progress: %s\n", PyString?_AsString(PyObject?_CallMethod(ds, "progress", NULL)));

return Py_BuildValue("[i, O]", 5, Py_False, NULL); }

//Then a part of my main function

//...

download = PyObject?_CallMethodObjArgs(session, PyString?_FromString("start_download"), tdef, dscfg, NULL);

if (download) {

printf("ok, downloading\n");

//Python code I'd like to reproduce here:

// download.set_state_callback( download_state_callback );

//In C, it should be something like:

// PyObject?_CallMethodObjArgs(download, PyString?_FromString("set_state_callback"), callback, NULL);

}

//...

Anyone has an idea how I can pass my C function to Python?

P.S.: sorry about indentation (and I know you Python guys sure love this) but this forum makes hard to format code correctly.

Reply Quote

Message #137

I have no experience with that. You can check how the VLC python bindings work (see VLC source package, bindings/mediactrl-python) or how M2Crypto works, that is a python module that wraps OpenSSL.

Reply Quote

arno

07/03/08 16:34:18
- Message #139
  
  Thanks for the tip arno.
  
  To sum up my problem, I was trying, from C, to tell Python to call a C function.
  
  VLC didn't exactly do that (in fact I haven't find anyone who has done it yet) but it has helped me find a solution. From my C code, I write Python a binding to my C callback function, then I load this binding in the embedded Python interpreter, get the Python function in a C variable (PyObject? *) that I give back to Python a few lines further like this :
  
  download = PyObject_CallMethodObjArgs(session, PyString_FromString("start_download"), tdef, dscfg, NULL);
  
  PyObject_CallMethodObjArgs(download, PyString_FromString("set_state_callback"), dlCallback, NULL)
  
  When I tell Tribler what function to call back (with the last line above), it looks like it says "sure, I'll do this" (returned value is 1, which means that it went well). But then it doesn't.
  
  Now I'm trying to figure out why. Any idea how I can achieve this?
  
  Reply Quote
  
  Alexandre Franke
  
  07/08/08 17:32:24
- - Message #140
    
    To sum up my problem, I was trying, from C, to tell Python to call a C function. VLC didn't exactly do that (in fact I haven't find anyone who has done it yet) but it has helped me find a solution. From my C code, I write Python a binding to my C callback function, then I load this binding in the embedded Python interpreter, get the Python function in a C variable (PyObject? *) that I give back to Python a few lines further like this :
    
    Calling C from Python is actually quite common. All libraries actually do that :) Note that it does not matter for such constructs that you've started Python from within C.
    
    download = PyObject_CallMethodObjArgs(session, PyString_FromString("start_download"), tdef, dscfg, NULL); PyObject_CallMethodObjArgs(download, PyString_FromString("set_state_callback"), dlCallback, NULL)
    
    Casting pointers to booleans and passing them as objects surely can't be right.
    
    When I tell Tribler what function to call back (with the last line above), it looks like it says "sure, I'll do this" (returned value is 1, which means that it went well). But then it doesn't. Now I'm trying to figure out why. Any idea how I can achieve this?
    
    To call C from Python, simply shape your C function like a Python function, see the example half way in http://docs.python.org/api/arg-parsing.html. It can also be useful to look at the output of Pyrex and SWIG, both tools used to automatically create bindings between C and Python. The M2Crypto library does this, and so do the VLC Python bindings.
    
    Reply Quote
    
    jjdmol
    
    07/08/08 18:58:44
  - - Message #141
      
      Hmm your code in my e-mail contained ! before the function names, but I now see you used that to avoid making links out of them, sorry for the confusion :)
      
      Reply Quote
      
      jjdmol
      
      07/08/08 19:02:00
    - - Message #142
        
        Hmm your code in my e-mail contained ! before the function names, but I now see you used that to avoid making links out of them, sorry for the confusion :)
        
        Indeed :)
        
        In fact the software used to run this forum doesn't seem very appropriate for code excerpts, which is a shame for a software project :(
        
        Reply Quote
        
        Alexandre Franke
        
        07/08/08 21:52:05
    - Message #143
      
      Calling C from Python is actually quite common. All libraries actually do that :) Note that it does not matter for such constructs that you've started Python from within C.
      
      Not quite. Using Python or Python/C doesn't imply exactly the same behaviours as you have to juggle with variables and functions that you get from Python to C and then back to Python if you are in a Python/C context.
      
      When I told that I haven't seen anybody doing this yet, I was talking about going back and forth between C and Python (i.e. using Python/C), which to me seems a little more difficult than just using one language from another :)
      
      All libraries actually do that :)
      
      I am of course aware of that :)
      
      Casting pointers to booleans and passing them as objects surely can't be right.
      
      As you said in next reply, that was a formatting trick for the forum ;)
      
      To call C from Python, simply shape your C function like a Python function, see the example half way in http://docs.python.org/api/arg-parsing.html.
      
      Of course. The C prototype of my callback is:
      
      static PyObject * tribler_integration_download_state_callback(PyObject *self, PyObject *args)
      
      Then I have the following array:
      
      static PyMethodDef tribler_integration_methods[]
      
      And the following function:
      
      PyMODINIT_FUNC init_tribler_integration (void)
      
      See, I have read the doc :)
      
      So, as I said in a previous post:
      
      > > When I tell Tribler what function to call back (with the last line above), it looks like it says "sure, I'll do this" (returned value is 1, which means that it went well). But then it doesn't.
      
      Returned value is 1. That is, set_state_callback doesn't rant when I give him my binding as an argument. Therefore I assume it is well formed.
      
      The problem is that the printf I have in this callback function are never shown, so I think that this function isn't called, for whatever reason. My question is, what is the reason?
      
      It can also be useful to look at the output of Pyrex and SWIG, both tools used to automatically create bindings between C and Python. The M2Crypto library does this, and so do the VLC Python bindings.
      
      My code was indeed inspired by the VLC bindings.
      
      Reply Quote
      
      Alexandre Franke
      
      07/08/08 22:19:06
    - - Message #145
        
        Just a thought:
        
        1. printf's can get lost. Use os.mkdir() or something to test if it is really called.
        
        2. Why don't you pass a Python callback function to set_state_callback that then in turn calls your C function?
        
        Reply Quote
        
        arno
        
        07/19/08 07:18:48
      - Message #149
        
        Thanks for your reply.
        
        First, here's my code. I think it will be easier it you can see it all.
        
        http://www.pastebin.ca/1078411
        
        Second, about printf getting lost, as you can see in my code, I have another function (simple_test) with Python binding (simpletest) which I call a bit before set_state_callback. I can see the sentence in the console, this does show me that the binding stuff does work.
        
        The "system" call in tribler_integration_download_state_callback is there to do what you wanted me to test with os.mkdir(). And it doesn't work. If I copy the same line in simple_test, this does work. So I'm pretty sure that my callback never gets called back.
        
        Why don't you pass a Python callback function to set_state_callback that then in turn calls your C function?
        
        If I understand your question well, I think that it's exactly the same as what I am doing currently.
        
        I've also tried to add some lines in Tribler so that I can figure out what happens.
        
        (Number 1078428 on pastebin.ca, the forum won't allow me to submit a post with two links)
        
        But I don't see these in the console so I'm a bit confused.
        
        I'd like to be able to debug the Python code, to run it step by step, but when called from C, that doesn't seem easy.
        
        Reply Quote
        
        Alexandre Franke
        
        07/21/08 16:55:43
        
        Message #150
        
        Your code never actually lets the Python interpreter perform its work. You need to release the global interpreter lock (GIL) and reacquire it around the sleeps.
        
        I also highly recommend writing most of your Python code in actual Python instead of calling the API functions. It will make it much easier to read, and harder to introduce bugs with references etc.
        
        Reply Quote
        
        jdmol
        
        07/21/08 18:25:08
        
        Message #180
        
        Your code never actually lets the Python interpreter perform its work. You need to release the global interpreter lock (GIL) and reacquire it around the sleeps.
        
        Thanks for the hint on this topic, I was not aware of the GIL. This will need some investigation.
        
        I also highly recommend writing most of your Python code in actual Python instead of calling the API functions. It will make it much easier to read, and harder to introduce bugs with references etc.
        
        Indeed, mixing Python and C is quite hard the way I do it. It appeared to me that you either do it this way or use PyRun_SimpleString. Is PyRun_SimpleString what you are talking about? This would be far easier but I don't see how I would then be able to make it call a C function of mine to display feedback on the download progress (and I can't do it otherwise, this display function has to be C).
        
        Reply Quote
        
        anonymous
        
        08/11/08 16:45:06
Message #181

I thought about this whole issue and had another idea. We could have Tribler run on its own (i.e. not use the API) and have DBus interfaces that my C programme could use to communicate with Tribler. What do you think about it?

Reply Quote

Alexandre Franke

08/11/08 16:45:54
- Message #182
  
  http://dbus.freedesktop.org/doc/dbus-tutorial.html
  
  Hmm D-bus looks very interesting, it is a system for interprocess communication (IPC) which i guess uses its own layer to send messages between applications. And it seems to have interfaces for C and Python.
  
  This brings back some memories, people who have programmed on commodore Amiga computers know that all internal communications went through system "message ports" Which means if you wanted to communicate with the parallel port or serial port or sound device or other programs you send messages. Or if you would open a window, the system would send messages to your program with events. Most programs had message ports, and often had small "messenger" programs which you could run in shell or from cron to give programs commmands or just tell them to quit. Also if you would make some database program you just gave it a message port, and all programs who needed to access the database used its message port.
  
  Now .. i know Amiga people who progressed to Windows or Linux often still use something close to message ports .. but they use TCP/IP sockets to implement them. Which is actually very close to the Amiga system, with this difference that processes now can be on different machines, so i guess you need to check if client/server IP numbers are the same for security.
  
  So why not simply use TCP/IP for interproces communication.
  
  So imagine tribler acting like a server and having a TCP/IP port through which other software could communicate with it, like a TCP/IP bridge between python and C or other software. Now to "command" tribler you only need to setup a socket in any language and you can communicate .. exactly like you would access a HTTP server .. of cause you would need to change port number and message content.
  
  Now i have written a number of server/client systems or software to access http server using tcp/ip in C .. but never in python. So i cant make the python bit .. but i could provide some C code to access the port of a program. I think if tribler had a TCP/IP command port if would solve most of issieus people have with python as most languages have bsd sockets which you could use.
  
  You just need to agree over a packet format to use, do you use text as with a http server, or binary as with a database server. What commands do you want to give tribler, what information do you want to get back.
  
  Reply Quote
  
  AP
  
  08/15/08 17:28:01

Forum Overview > Tribler Forum > Issue with Tribler integration in C code

Search the Forum

Latest news

29 April 2008

17 February 2008