series2 OSD kind of working, but why does osdwriter fork?

[TheWickedPriest]

Originally posted by embeem
1st: whoever keeps bitching about the screen module, knock it off;

No one was bitching about it; he just didn't get how it was supposed to work.

2nd: using a png and osdwriter is kludge

Duh. However, it works, and works now. When there's something better, that'll be great, but it's silly to say that we shouldn't start with the easy way just because it's a kludge.

[mmccurdy]

Oh wow, we all thought osdwriter was as efficient as it gets since it requires converting text to an image then killing the app afterwards. Cut us some slack. If we had the rfc so-to-speak for OSD on a series 2, we'd be working that way. Until then, I want the easy fix.

If you can add details, we're all for it.

[TheWickedPriest]

Originally posted by Xybyre
Next, I modified fly to create truecolor images with the alpha channel enabled (it currently only creates paletted images).

Xybrye, could you please post your patches for fly to accomplish this? I get good images with the stock fly, until I enable transparency; then the palette is all screwed up. It works for white on black (= transparent), but nothing else. Using size 708x480 helps, but not enough.

Another useful mod might be to save the PNGs without compression; but that would have to be done in gdlib.

The largest font that comes with gdlib is still small when viewed on the TV.

Yeah. It looks like these should be easy to build -- see gdfont*.c. The comments say they were created with "bdftogd". I found that; now to find some BDF fonts. :-) I do have one that's 11x19, a little bigger than "giant".

Or I could rebuild gdlib with the TTF library...

It would be neat to be able to use the TT fonts already present, for a unified look.

Third (ok, I don't count good), I could write a new 'text2osd' that emulates the one for series1. However, I don't know what the existing one does, or how it looks...

Me neither. Anybody got screenshots of this?

My very minor contribution: Here's my current modification of mmccurdy's newtext2osd:

#!/bin/bash
#
# Here are the original Series 1 newtext2osd arguments
# newtext2osd -s seconds -w file -f color -b color -d file
# newtext2osd -s seconds -f color -b color -x xpos -y ypos -t text

if [ ! -n "$2" ]; then
  echo "Usage: newtext2osd \"<string to display>\" <length to display>"
  exit;
fi

cd /var/tmp

text=$1
len=$[9 * ${#text}]
offset=$[(708 - $len) / 2]

cat <<EOF | fly -q -o tmp.png
new
size 708,480
type png
fill 0,0,0,0,0
frect $[$offset - 4],228,$[$offset + $len + 4],251,200,200,200
string 0,0,0,$offset,232,giant,$text
transparent 0,0,0
EOF

print tmp
sleep $2
clear

This puts the text inverted in a box in the center of the screen. It doesn't work quite like I first intended, because of the palette issues (I had to change the colors); but there are some points of interest: one less temp file created, and the centering scheme. This is probably about as far as I'd want to go with a shell script. :-) I'd do what you did and build directly on gdlib, but I don't have a cross-compiler set up right now.

Oh, and "print" is this separate script I made earlier:

osdwriter $1.png -share
kill `ps x|grep osdwriter|grep -v grep|cut -b -5`

while "clear" is this:

osdwriter /tvbin/nothing.png -share
kill `ps x|grep osdwriter|grep -v grep|cut -b -5`

I probably should integrate those, or at least "source" them instead of calling them.

[TheWickedPriest]

OK, here are some bigger fonts I made for libgd and fly. These are taken from vga11x19.bdf, 10x20.pcf, and 12x24.pcf. Not tested on my Tivo since I still don't have the cross-compiler set up, but they work great on my x86 Linux box. Note: I'm using the latest gd, 2.0.15, rather than 1.8.4.

[TheWickedPriest]

No takers, eh? Yet, that stupid "Almost there" screen I posted in another thread shows 35 downloads. Go figure...

Anyway, here's a small refinement to the script -- no more temp files:

#!/bin/bash

killosd() {
  kill `ps x|grep osdwriter|grep -v grep|cut -b -5`
}

if [ ! -n "$2" ]; then
  echo "Usage: newtext2osd \"<string to display>\" <length to display>"
  exit;
fi

text=$1
len=$[9 * ${#text}]
offset=$[(708 - $len) / 2]

cat <<EOF | fly -q | osdwriter /proc/self/fd/0 -share
new
size 708,480
type png
fill 0,0,0,0,0
frect $[$offset - 4],228,$[$offset + $len + 4],251,200,200,200
string 0,0,0,$offset,232,giant,$text
transparent 0,0,0
EOF

killosd
sleep $2
osdwriter /tvbin/nothing.png -share
killosd

[Xybyre]

TheWickedPriest, thanks for verifying the problem with indexed PNGs even at 708x480. I was seeing it too, but had not done enough testing to be sure. I had rewritten my yac client to create an indexed image, and couldn't see the black box around my white text.

Fly is easy enough to modify to create truecolor images. One of my previous posts listed the two functions needed. However, I'm in the middle of some modifications, so I can't create a usable patch right now. I'm modifying it to allow use of TrueType fonts.

I built a gdlib that supports TrueType fonts, and it works well with YAC to display an easily-readable caller-id box on the TV. However, yac seems slow at times, either because I'm using truecolor, or because I'm using TrueType fonts (or both). Sometimes I don't get the caller id box until after the third ring. I will try it again using your fonts, to see if it improves performance.

To anybody who wants to help me test this, please PM me.

thanks,
Xybyre

[Xybyre]

Okay, I've found the cause of the performance problem. It is the saving of the true-color PNG file that is taking a long time.

It takes 2.6 seconds to create and save a true-color PNG file with an alpha channel. It takes 0.3 seconds to create and save an indexed PNG file. Just the saving part takes 2.5 seconds for the true-color PNG. Saving with no compression for the PNG does not seem to help.

I takes almost no time at all to create the image itself (0.1 seconds). Using a TrueType font adds about 0.2 seconds.

Looks like we are stuck with white-only for the caller id info unless we can figure out how to save the file faster...

[mmccurdy]

Xybyre, please post your yac.c changes. I didn't see any external calls to newtext2osd like I expected to see in the source. I appreciate everyone's work on getting OSD working. Now if we can only do it like MyWorld does it!

[TheWickedPriest]

Originally posted by Xybyre
It takes 2.6 seconds to create and save a true-color PNG file with an alpha channel. It takes 0.3 seconds to create and save an indexed PNG file. Just the saving part takes 2.5 seconds for the true-color PNG. Saving with no compression for the PNG does not seem to help.

I've just been reading PNG: The Definitive Guide, and I think I've found the source of this: it's the "filtering" step, which is applied before zlib compression. It's not done in paletted images, only in truecolor. If we can turn that off, it might do the trick.

[Xybyre]

Originally posted by TheWickedPriest
I've just been reading PNG: The Definitive Guide, and I think I've found the source of this: it's the "filtering" step, which is applied before zlib compression. It's not done in paletted images, only in truecolor. If we can turn that off, it might do the trick.

Excellent work! Disabling filtering in gdlib shaves off another second. Now it takes 1.6 seconds to save a truecolor png. Disabling compression saves another 0.10 seconds, so that doesn't make much of a difference (but we'll take what we can get).

A 1.3 second penalty for truecolor may be tolerable for most of us, but there may be more things we can do to optimize further. Do you have any other suggestions? In the meantime, I'll start reading through that book myself...

Also, I finally figured out why osdwriter forks! It needs to call setsid(). The setsid() man page states that the process must fork and call setsid() from the child process. That doesn't help with the hang problem, but now I know.

On a more interesting note, while I was tracing osdwriter, I discovered that it was hanging on the _newselect() function. I wrote my own select() to do an exit(0), put it in a shared library, and ran "LD_PRELOAD=select.so osdwriter myfile.png -share" (thanks to David Bought for this trick), and now osdwriter terminates without me having to kill it! (I also tried returning 0 and -1, but then select() got called in an infinite loop)

We are getting close to a usable solution now!

[Xybyre]

After spending more hours on this, I have managed to optimize gdlib truecolor PNG saving to acceptable levels!

The conversion from gdlib's internal format to PNG image format went from 0.30 seconds to 0.15 seconds. Not too much of an improvement, but every bit helps.

The actual file saving went from 1.3 seconds to 0.3 seconds! Digging through the documentation, I found some functions for setting the libpng zlib compression options. Using no compression saved 0.2 seconds, but using the fastest compression setting saved 0.5 seconds. I also changed the compression memory level and window bit size, which helped a lot.

Long story short, yac now takes 0.5 seconds to display caller id info! Using a TrueType font adds about 0.2 seconds. I did try the fonts posted by TheWickedPriest, but they were too skinny, and didn't look good, IMHO.

I have a fly that can create true-color images and allows the use of TrueType fonts. The bad news is that stupid me did a "ln -s ../fly2/fly.c fly.c" in ./fly1, when ../fly2/fly.c was linked to ../fly1/fly.c. doh! Now my source code is gone. ./fly1 had my MIPs environment, and ./fly2 had my native linux environment. Argh!! why doesn't "ln" warn you?!! <sigh>

I'm pretty happy with what I have now, so I'll clean up the code (mostly removing debug statements) and post my binaries shortly.

[mmccurdy]

Great work. I've been too swamped to really pour time into this for awhile. I'm left with a compiled elseed app that picks up and displays the caller id along with a working albeit inefficient fly-calling script. My problem is getting the binary to call the OSD script and avoiding a segmentation fault. blah!

Looking forward to your binaries and source, Xybyre.

[Xybyre]

You can download my distribution package for libgd, yac, and fly at http://www.xybyre.net/tivo/

Installation and usage instructions are in the readme.txt.

libgd contains some speed optimizations. Specifically, filtering is disabled, PNG compression options have been tweaked, and pointer arithmetic was used to replace array element references.

yac has been modified to create true-color images with an alpha channel, and it calls "osd" to display the generated image.

"osd" is my shell script for displaying a PNG using osdwriter. It preloads a shared library to replace the select() function in osdwriter so that osdwriter quits instead of hanging.

"fly" has been modified to support the creation of truecolor images as well as TrueType fonts. fly is not needed by yac or any of its libraries. This is just thrown in as a bonus.

I'll post my source code patches later when I have some time.

[Fletch319]

Let me be the first to congradulate you on your work! I know a lot of people have missed the text2osd on the series1. Although this may be an odd way to solve the issue it definately beats nothing. I do have one question. I know you need a YAC server to provide the callerid information. Can we use the tivo as the yac server as well? Although i never had a series 1 i believe that's what elseed was correct? Of course many of us have the phone lines disconnected so you would need to make a special cable described in other threads that prevents dialout but still allows callerid info in.

Can we adapt this solution so the tivo is the YAC server as well?

Thanks again for all your hard work Xybyre!

[TiVoByte]

I have been testing this for a few days, all I can say is GREAT WORK. Once you have the caller ID habit, you really need your fix.
My S1 had yac running for quite some time.

Also, I'm not the sharpest knife in the drawer, Xybyre took the time to help me with my ineptness. Great guy. The distribution is dork (me) proof.

Fletch, I don't think server software was ever released for the Tivo. Don't you have at least one PC on all of the time that can act as the server? I can count 5 in my house that are on all of the time, not counting any test PC's.

Robert

P.S. YAC Rocks!