Rendering images asynchronously

Hello List,

I need to generate a bunch of thumbnails, and since it takes some time, I would like to do it in another process, so that the GUI is not blocked while they are generated. I'm using GP to draw the images, which means that the image objects used for the thumbnails need a CAPI pane to be initialized. And the CAPI pane needs to be accessed in the correct process, which on MacOSX is the Cocoa Event Loop, i.e. the GUI process.

As far as I can tell, this makes it impossible to render images "in the background" using GP (at least on Cocoa, on Windows, it would probably be possible to display a dummy interface in its own process and using that to create the image objects).

Is there any way around this? (Of course I could use some external graphics library instead of GP, but it's not a very attractive solution)

Regards
Erik



_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: Rendering images asynchronously

Take a look at gp:pixmap-port to see if it suits your needs.

pt

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: Rendering images asynchronously

Erik,

They definitely need to be rendered in the pane process. I don't know that they need to be created there.. But, assuming they do, the majority of time won't be generating the thumbnail in memory instead of loading the source image that's used to generate the thumbnail off disk (assuming they are images).

I'd recommend something like giving your interface a mailbox and the thread that loads sources a mailbox as well. The load thread is just waiting on #'mp:mailbox-read for a filename (or whatever it is that you use as source for the thumbnail) to be written to it, read the source, and then send it to the pane process mailbox, which in turn is periodically checking and using that data to create the thumbnail.

Finally, I don't know if you have some large images that you are using to create smaller ones or if your source is something entirely different (e.g. a 3D model that needs to be rendered before being captured as a thumbnail image). I'm assuming that it's just a larger image you want to render smaller. If that's the case, you may want to just use gp:with-graphics-scale instead of actually making a smaller version of the image.

HTH. If my assumptions were way off base, then more information is needed. ;-)

Jeff M.

On Thu, Jul 4, 2013 at 7:26 AM, Erik Ronström <erik.ronstrom@doremir.com> wrote:

Thanks! Well, yes, I am using that already. But it seems that it leaks memory if run in a different process than the pane it was created with, which suggests that it should be run with apply-in-pane-process.

Or am I wrong? In that case I need to find the correct way to do it!

Regards
Erik

objc[4430]: Object 0x1247f0 of class __NSArrayI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0x1247f0 of class __NSArrayI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0xe0c11f0 of class __NSArrayM autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0x1247f0 of class __NSArrayI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0xe0d7e10 of class __NSArrayM autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0x1247f0 of class __NSArrayI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug

4 jul 2013 kl. 14:23 skrev Paul Tarvydas:

> Take a look at gp:pixmap-port to see if it suits your needs.
>
> pt
>

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html

Re: Rendering images asynchronously

Erik,

"Render" == "draw". :-)

By "I don't know" I meant to imply that I didn't know if they needed to be *created* in the pane's process. Since you are drawing multiple things to a port in order to generate your thumbnail, then yes, that part does need to take place in the pane's process. I thought that perhaps you were doing something different.

At this point, my only suggestions (others may have better options) would be one of the following:

1. Have a thumbnail generation queue that is periodically processed to continuously generate the thumbnails from within the pane process. Each entry in the queue would likely either some data used to draw the next piece of a thumbnail (the pixmap port and the operation) or just a closure that does it.

Pseudo code:

(defvar *thumbnail-queue*)

(defun insert-thumbnail-operation (function)

(enqueue *thumbnail-queue* function))

(defun process-thumbnail-queue (pane)

(apply-in-pane-process pane

(loop for a few milliseconds...

(let ((op (dequeue *thumbnail-queue*)))

(funcall op)))))

(defun create-example-thumbnail (pane port)

(insert-thumbnail-operation #'(lambda () (gp:draw-line port 0 0 100 100)))

(insert-thumbnail-operation #'(lambda () (gp:draw-circle port 50 50 10)))

(insert-thumbnail-operation #'(lambda () (notify-pane-thumbnail-complete pane port))))

I'm not sure if this solution will have issues with GP state changes (e.g. setting the color or thickness of the line you'd like to draw). But most GP function calls allow you to pass those options as keywords instead of modifying the global state before the call.

2. Use something other than LispWorks to generate the thumbnails. For example, you can wrap ImageMagick (http://www.imagemagick.org/) use FLI and use similar functions for the operations you want, but just in another thread. Then convert the ImageMagick, in-memory image to a pixmap-port when done (I don't know how to do this, but I'm sure others here have done it).. This likely requires a higher up-front cost in creating the FLI wrappers if someone else here hasn't already done it.

HTH,

Jeff M.

On Thu, Jul 4, 2013 at 9:25 AM, Erik Ronström <erik.ronstrom@doremir.com> wrote:

Thanks for your answer!

You write "They definitely need to be rendered in the pane process", but just to make it clear: what do you mean by "rendered" in this case? If it's "to draw the in-memory pixmaps to the output-pane", then yes, that should definitely be done in the output-pane's process. But that is a very fast operation, especially if the images are already loaded (using load-image) to the pane.

So what I'm interested in is the generation of the in-memory pixmap images, which is also what takes time. And the source is not a big image, but rather a quite long list of objects that draw themselves to a port using different gp instructions (draw-line, draw-string etc). So it would not be an easy task to separate the drawing logic (which supposedly takes more time) from the drawing instructions (supposedly taking less time).

But the core problem is that I cannot generate the in-memory images in another thread than the interface's thread (which on MacOSX is the Cocoa Event Loop), and this is because the pixmap-port needs to be attached to a pane, and therefore also needs to be accessed in that pane's process.

I don't know that they need to be created there.

Do you mean that you don't know, or that you know that they don't need to? (Maybe you were clear, but I'm not a native English speaker :)

If you know it's possible to use pixmap-port in another process, that's very nice to know! In that case I should try it again, and I also would be very thankful for some example of how it can be done!

Regards
Erik

4 jul 2013 kl. 16:03 skrev Jeff Massung:

Erik,

They definitely need to be rendered in the pane process. I don't know that they need to be created there. But, assuming they do, the majority of time won't be generating the thumbnail in memory instead of loading the source image that's used to generate the thumbnail off disk (assuming they are images).

I'd recommend something like giving your interface a mailbox and the thread that loads sources a mailbox as well. The load thread is just waiting on #'mp:mailbox-read for a filename (or whatever it is that you use as source for the thumbnail) to be written to it, read the source, and then send it to the pane process mailbox, which in turn is periodically checking and using that data to create the thumbnail.

Finally, I don't know if you have some large images that you are using to create smaller ones or if your source is something entirely different (e.g. a 3D model that needs to be rendered before being captured as a thumbnail image). I'm assuming that it's just a larger image you want to render smaller. If that's the case, you may want to just use gp:with-graphics-scale instead of actually making a smaller version of the image.

HTH. If my assumptions were way off base, then more information is needed. ;-)

Jeff M.

On Thu, Jul 4, 2013 at 7:26 AM, Erik Ronström <erik.ronstrom@doremir.com> wrote:

Thanks! Well, yes, I am using that already. But it seems that it leaks memory if run in a different process than the pane it was created with, which suggests that it should be run with apply-in-pane-process.

Or am I wrong? In that case I need to find the correct way to do it!

Regards
Erik

objc[4430]: Object 0x1247f0 of class __NSArrayI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0x1247f0 of class __NSArrayI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0xe0c11f0 of class __NSArrayM autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0x1247f0 of class __NSArrayI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0xe0d7e10 of class __NSArrayM autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[4430]: Object 0x1247f0 of class __NSArrayI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug

4 jul 2013 kl. 14:23 skrev Paul Tarvydas:

> Take a look at gp:pixmap-port to see if it suits your needs.
>
> pt
>

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
lisp-hug@lispworks.com
http://www.lispworks.com/support/lisp-hug.html