(Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

Greetings from a lovely fall day in the Rocky Mountains...  The Aspen 
trees are turning color :)

In my two-years of building CAPI/GP-based software, I have constantly 
struggled to get areas to update/redraw on my command via 
gp:invalidate-rectangle.

Usually it happily updates, but about a third of the time various 
contortions are required to get an update. Sometimes, I throw my 
hands up; trash the piece of software under construction; and just 
start over from scratch - and eventually get the thing to update by 
some ugly contortion or the other.

I used to believe that I was simply more obtuse than most and just 
did not understand the fundamentals of the thing, but considering the 
remarkable workaround suggestions that I have received from Lisp-HUG 
(that actually work - ugly as they are), I no longer believe that, at 
least not completely :)

Spawning threads to do the update complete with message-receiving 
mailboxes and all the rest - Good Heavens!

The whole thing seems really quite arbitrary.

Question: Can someone suggest a reliable paradigm to get reliable 
redraws on command via gp:invalidate-rectangle - a set of rules that 
provides such reliability?????

If such a set of rules exist, I would love to see what they are - 
80%+ of my CAPI/GP problems would simply go away if I knew what such 
a set of rules might be.

Thoughts?

Regards to the List.

Jack Harper ("Redraw me as still in the dark" :)
Secure Outcomes Inc.
Evergreen, Colorado USA

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

I am somewhat perplexed by your request... GP:INVALIDATE... simply accumulates damage regions that will be redrawn the next time a redraw actually occurs. I don't think I would want the screen to actually start performing updates immediately, since there are frequently many interrelated regions that will accumulate damage.

The model for redrawing appears to vary widely across OS's, with Windows behaving differently from OS X, and who knows how X-windows or Linux behaves? Getting smooth updates has been my quest, and for Windows that usually means performing all updates in a hidden buffer and then rapidly scribing that buffer to the screen. OS X behaves much more rationally than this, with the double buffering still working, but usually not needed.

Why would you want to dictate to the underlying OS when to actually perform the screen redraws? (unless the OS is insufficient in this regard -- perhaps so on Windows)...

Dr. David McClain

Chief Technical Officer

Refined Audiometrics Laboratory

4391 N. Camino Ferreo

Tucson, AZ 85750

email: dbm@refined-audiometrics.com

phone: 1.520.390.3995

web: http://refined-audiometrics.com

On Sep 20, 2010, at 10:17, Jack Harper wrote:

Greetings from a lovely fall day in the Rocky Mountains... The Aspen trees are turning color :)

In my two-years of building CAPI/GP-based software, I have constantly struggled to get areas to update/redraw on my command via gp:invalidate-rectangle.

Usually it happily updates, but about a third of the time various contortions are required to get an update. Sometimes, I throw my hands up; trash the piece of software under construction; and just start over from scratch - and eventually get the thing to update by some ugly contortion or the other.

I used to believe that I was simply more obtuse than most and just did not understand the fundamentals of the thing, but considering the remarkable workaround suggestions that I have received from Lisp-HUG (that actually work - ugly as they are), I no longer believe that, at least not completely :)

Spawning threads to do the update complete with message-receiving mailboxes and all the rest - Good Heavens!

The whole thing seems really quite arbitrary.

Question: Can someone suggest a reliable paradigm to get reliable redraws on command via gp:invalidate-rectangle - a set of rules that provides such reliability?????

If such a set of rules exist, I would love to see what they are - 80%+ of my CAPI/GP problems would simply go away if I knew what such a set of rules might be.

Thoughts?

Regards to the List.

Jack Harper ("Redraw me as still in the dark" :)
Secure Outcomes Inc.
Evergreen, Colorado USA

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

At 11:33 AM 9/20/2010, David McClain said:

I am somewhat perplexed by your request... GP:INVALIDATE... simply accumulates damage regions that will be redrawn the next time a redraw actually occurs. I don't think I would want the screen to actually start performing updates immediately, since there are frequently many interrelated regions that will accumulate damage.

The model for redrawing appears to vary widely across OS's, with Windows behaving differently from OS X, and who knows how X-windows or Linux behaves? Getting smooth updates has been my quest, and for Windows that usually means performing all updates in a hidden buffer and then rapidly scribing that buffer to the screen. OS X behaves much more rationally than this, with the double buffering still working, but usually not needed.

Why would you want to dictate to the underlying OS when to actually perform the screen redraws? (unless the OS is insufficient in this regard -- perhaps so on Windows)...<snip>

Thank You David for your response.

What you say makes sense - gp:invalidate-rectangle simply queues up areas to redraw once an actual refresh happens - whenever that is.

I suppose that my questions devolves to "Is there any way to force a refresh to occur to emit the queued-up area draws?"

I know one way that works absolutely every time I use it: a call to capi:display-message invariably causes my gp:invalidate-rectangle calls to really really happen on the screen. I use capi:display-message to see if, at least, the logic is correct - things are fine until I remove the capi:display-message and then screen updating does not (in that 1/3 of the cases) happen.

Anyway, to answer your question: An example is reading real-time data from a fingerprint scanning device running at video rates and displaying the frames to the screen so that fingerprint "rolls" appear fluid on the screen.

Waiting for CAPI > Windows to bother to emit the changed pixels simply doesn't cut it -

Is there any way to instruct CAPI to actually refresh the screen??

I can gp:invalidate-rectangle the area as many times as I want - but, nothing actually happens until something deep inside CAPI/Windows decides to actually spit out pixels. How can I force that?

Regards to the List.

Jack Harper
Secure Outcomes Inc.
Evergreen, Colorado USA

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....


><snip>
>
>Anyway, to answer your question: An example is reading real-time 
>data from a fingerprint scanning device running at video rates and 
>displaying the frames to the screen so that fingerprint "rolls" 
>appear fluid on the screen.<snip>


Follow-up:

What finally worked was spawning a separate thread that gathered the 
video data and wrote that into a double-buffering scheme etc etc etc.

Ugly, but it worked.

Best that I can tell, it works because the CAPI thread(s) is free to 
actually do a screen update.

BTW, the obvious 
capi:force-screen-update/capi:force-update-all-screens had no effect 
- I assume that those do nothing other than set a flag somewhere that 
the CAPI process(s) eventually see - so, they really don't "force" 
anything.  I really have no idea what they really do.

I wonder if 6.0 might help with all this...  Anyone care to comment?


Regards to the List.

Jack Harper
Secure Outcomes Inc.
Evergreen, Colorado USA

ps - Not being critical of CAPI & Friends - it is a great package - 
Just trying to understand.

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

> Greetings from a lovely fall day in the Rocky
> Mountains...

Greetings from Moscow! :)


> Question: Can someone suggest a reliable paradigm ... [skipped]

To me the reliable paradigm is to get down to the Win32 API. CAPI is great, but with direct call to winapi I can accomplish more with less pain.

See

http://paste.lisp.org/display/114733


That's a little code that I pulled together from my previous tests. Run (chug-chug) and press the redraw button. It redraws the entire draw pane. (chug-chug-chug) is the same as (chug-chug) but after the start it redraws itself every 1 second, but you can still press the redraw button to redraw between pauses. (win32:invalidate-rect ..) is an equivalent for InvalidateRect (see MSDN for that).

Best,
 Art

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

At 04:35 PM 9/20/2010, Art Obrezan wrote:

> > Greetings from a lovely fall day in the Rocky
> > Mountains...
>
>Greetings from Moscow! :)
>
>
> > Question: Can someone suggest a reliable paradigm ... [skipped]
>
>To me the reliable paradigm is to get down to the Win32 API. CAPI is 
>great, but with direct call to winapi I can accomplish more with less pain.

Art - WOW!

I have quite a bit of experience building C/C++ software (Yuck!) over 
the years with the WIN32 API - Such a move would clearly get rid of 
many problems...

The only downside that I see would be that the Lisp application 
would, of course, then be platform-dependent.

I plan to move our system to Linux in about a year - this is paced by 
availability of special hardware device drivers and the like.

Neat idea though - and worth contemplation here amongst the Aspen :)

Regards,

Jack Harper
Secure Outcomes Inc.
Evergreen, Colorado USA

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

> The only downside that I see would be that the Lisp
> application would, of course, then be platform-dependent.

To me, platform-independency is almost a myth :) You can get close to such independency, but never reach it.

> I plan to move our system to Linux in about a year - this
> is paced by  availability of special hardware device drivers
> and the like.

No problem. See

http://paste.lisp.org/display/114739

It's for linux, LW6. The same chug-chugs with access to gtk' "low-level" functions. The idea is the same as in the win32 version, I just changed a few lines.

Best,
 Art

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

Unable to parse email body. Email id is 10577

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

Unable to parse email body. Email id is 10576

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

Unable to parse email body. Email id is 10578

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

At 01:28 PM 9/20/2010, David McClain wrote:
>Hi,
>
>Not sure if you indicated Windows or other OS? But on my OS X, just 
>using straightforward CAPI (including the offscreen buffer update to 
>be compatible with the same source between OS X and Windows), I'm 
>seeing typically 10-20 Hz screen updates on an older MacBook, and 
>even faster on a more recent iMac. And that is without doing 
>anything to force the redraw on screen.
>
>Now realtime video is a tougher nut to crack, since you have some 
>kind of frame buffer attached to the camera hardware. But if I 
>recall correctly, even typical webcams can only update around 10 Hz. 
>Just the same, unless it is a simple web interface, you probably 
>have hardware interrupts and hardware drivers to contend with.
>
>In the past, I let those kinds of windows be run by the lower level 
>high-performance code, but attached the foreign window to a Capi 
>frame so that I could incorporate those images in the Capi GUI that 
>I was building. I have done this for sure on astronomical imaging 
>CCD's, but those weren't high-speed readouts.



Yes, David, you are correct - the fingerprint scan device is running 
at about 10 frames per second.   ..And yes, there are hardware 
interrupts and drivers and all of that wandering about on the other 
side of the FFI to get the real-time data into Lisp Space.  Great fun 
actually :)

Regards to the List.

Jack Harper
Secure Outcomes Inc.
Evergreen, Colorado USA

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

At 10:09 PM 9/20/2010, Chris/U.K. wrote:<snip>
>Hi Jack,
>
>I had a problem a while back relating to invalidation whilst scrolling (on
>WinXP). It seemed as if a redraw would only occur at the END of a 
>scroll event,
>if my drawing code was even mildly expensive. I eventually solved it by
>including a call to win32:update-window whenever the window was 
>scrolled - this
>forces immediate invalidation. WM_PAINT messages are low priority, apparently,
>and will get delayed if the message queue is sufficiently clogged.
>
>Chris

....Another Brick in the Wall.

Chris, that sounds very interesting and I will try it.

The idea of mixing CAPI with direct calls into WIN32 to actually 
force a redraw sounds like a good approach.

Thank You for the added data point.

Regards to the List.

Jack Harper
Secure Outcomes Inc
Evergreen, Colorado USA

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

At 05:12 AM 9/21/2010, Martin/Lisp HQ wrote:

><snip>
>
>BTW, why do you think it is ugly and do you have any suggestions as to how it
>can be improved?

Good Evening Martin - I appreciate your input as always.

"Ugly" was too strong a term - How 'bout "inelegant" or "clumsy"...

I find it "clumsy" primarily because it is not at all obvious how to 
do such a thing to the neophyte from reading the documentation.

It took me a long time to figure out how to approach it by spawning 
threads and all that and I still have trouble with it and still make 
such things work really by trying all combinations - "spit and baling 
wire". Once one understands the general idea, it becomes less 
"clumsy" simply because one become more used to it.

As to how to make it better...     Real preemptive multiprocessing 
would help - but *that* is a Tall Order in a Lisp system.

Another possibility would be to modify capi:force-screen-update to 
not just set a flag for some other process to notice but to actually 
shove pixels out to the screen from within the calling 
process/thread. Obviously, I don't know enough about the CAPI 
internals to make a judgement on that - CAPI sits, of course, on top 
of Win32, Linux, Mac, etc etc, which clearly makes things more obscure.

Regards,

Jack Harper
Secure Outcomes Inc.
Evergreen, Colorado USA

Re: (Lisp-HUG) Reliable Invalidate-Rectangle Redraws....

Unable to parse email body. Email id is 10584