Some contact list buddy icons appear pixelated in Leopard

*nod* I've been seeing that for a while in 10.5 dev builds; the cache of the images we're receiving from libpurple look fine in Preview, but pixelation occurs nearly everywhere the icon displays in Adium (tooltip, contact list, get info window).

Do the same pictures look correct in the message window?

Nope, they're pixelated in the message window as well. Definitely Leopard specific- everyone I know who upgraded is encountering it. It seems to happen on changed pictures, but old ones will remain OK. Happens in 1.2svn as well.

I'm also getting this issue on Leopard, using 1.1.3

iseem to have it only when they were logged on before me and only on msn

also when i resized the image tru prefs > apearance > customize list style, it was blurred even when no mouse over so i guess it has to do with the resize functions i'll check the core image info for 10.5 since the own display selector aint working either.

They only wind up pixelated if the original image is 4kb in size, and a JPG (however for some reason each time I delete the cache and it redownloads the display pictures, there is always ONE PNG that is the exception to the rule - but only one). The images do indeed show OK in the cache directory. All PNG files of 16 - 32kb in size look fine in both Adium and Preview (except that one exception).

Interesting Point: Get Info on a contact whose display picture is exhibiting this problem. Then double click their display picture in the Get Info window. The Apple Image Picker automatically adjusts it to it's 'correct' size (about half?) such that it is no longer pixelated.

This seems to have been solved at some point... in my previous notes here I mentioned that removing my icon cache and allowing it to re-download display pics did not solve the problem. Despite this, when contacts change their display picture or reset the old one, the problem goes away. Perhaps it has something to do with the rendering of images on Leopard that were processed previously in Tiger.

In every case (for the last couple days) when a contact changes their picture, the new ones are not pixelated. Can anyone else confirm this?

The very same contacts that show pixelated display pictures in Adium do not show any display picture at all in Microsoft Messenger. Those who 'changed/updated' their display picture so that it was no longer pixelated in Adium, show just fine in Microsoft Messenger.

I'm starting to think this was a temporary Microsoft server issue...

It was definitely on AIM that I saw this issue, so I don't think it was an MSN server problem.

Yeah, I'm seeing it across all services.

Can confirm this as well on 10.5 GM and myspace buddy icons. It definitely stretches across all services.

MysticalOS: Could you try getting the MySpace buddy to re-set their icon to see if it fixes the problem? I'm trying to figure out if that workaround works on all protocols or just MSN :)

Additional notes: Clearing the icon cache doesn't solve the problem (like I mentioned above) yet getting the contact to change their icon (even if they change it back) fixes it. The only thing that I know of that changes in that process (compared to just clearing the cache and allowing the icons to be re-downloaded) is the actual name of the buddy icon. I believe in MSN it's a seemingly random string - perhaps a hash.

This also fixed it for myspaceim as well, myspaceim is more of a pain to change since the icon is their default profile pic. user first has to change their profile pic, then, it still doesn't refresh until that user signs into myspaceim once, then their client refreshes the icon and sends new one...needless to say yes it did clear the pixel issue.

It might be informative to have take a contact with a blurry icon who knows where the original file is that is being used, save the cached version of that icon on our side(has it been determined if that cached version shows properly in Preview or another graphics file viewer?), have the contact change to a new icon, then have the contact change back to the original file. That may not be a feasible test to actually generate in the wild unless we have a way of producing blurry icons ourselves, though. I'm just wondering if it's a difference in the file being sent to us or in the way we're processing it...

It has been determined that the cached version shows properly in preview - no pixelation, even while it is still pixelated in the contact list. I have requested that contacts change their icon to a new one, then change it back to the original file, and it always works fine after doing so.

So my conclusion regarding those tests is that it can't be how the file is being sent because it works fine if the contact changes pics and/or puts it back to the original picture, and it displays fine in Preview. It only appears pixelated within Adium. Yet, deleting the local cache, forcing the redownload of the image from the server, results in pixelation again. It seems that the server copy must be updated before the problem is fixed.

Hmmm. fascinating!

Can we compare the cached version which looks pixelated in Adium to the updated version of the same icon which doesn't? Does a diff show any changes? What about metadata, etc? (Could you attach the two files here so anyone with any thoughts on what might be different can run tests?)

Sounds like this is related to #8012.

With the confirmation from MysticalOS, all pixelated images are 96 pixels / inch DPI whereas the ones that displayed OK were all 72 pixels / inch

dpi (not image size)

All clear: 72x72 71x71 28x28

All pixelated and bad: 96x96 180x182 150x150 96x96 95x95 100x100 230x230

180x180 typo

It appears everything 72 dpi and under is clear, anything above it distorts.

NSLogging the local ownSize variable in -drawRoundedInRect:atSize:position:fraction:radius: in AIImageAdditions reveals that the image has already been scaled to size when it gets there. Curious. Are we pre-scaling the icon somewhere?

[Forgot to post my previous comment earlier.]

Yes, we are: AIUserIcons.

AIUserIcons is a cache for pre-scaled user icons. When an icon is not found in the cache, it obtains the icon from the AIListContact by sending it userIcon.

The bug, so far, is that some contacts return a 16×16 icon from that method. You can see how it would be hard for Cocoa to make up pixels in a 50×50 icon from a 16×16 icon.

That may not be the only issue at play. I've made a simple test app which:

1. Loads a 300 dpi image and displays it in an NSImageView
2. Loads the same image and uses our drawing functions to draw it in a view.

The image is 170x135 pixels @ 300 dpi. Xcode displays it too small; NSImageView displays it too small; our drawing code displays it waaaay too small. Uncommenting

[image setSize:[image sizeInPixels]];

from -[ImageDpiTest awakeFromNib] makes the NSImageView display properly.

Uncommenting to two setSize: calls in -[NSImage(AIImageAdditions) drawRoundedInRect:atSize:position:fraction:radius: makes the view-with-drawing work properly.

Note that this AIImageAdditions.m uses the sizeInPixels method recently removed. Note also that it has an improved implementation of rectForDrawingInRect:atSize:position which does use sizeInPixels but would be improved even if using size, I believe - it scales the returned rect appropriately if the rect is too small to actually display the desired size.

300 dpi image display test app

So that's what I've got - back to you, Peter :)

Oh, one more comment (sorry for the ticket spamming). Doing these "fixes" in Adium itself seemed to lead to aberrant behavior, presumably from the multiple changes in size causing permanent changes to the image despite setting the size back after acting. I have no idea why :(

NSLog(@"%s: Contact %@ has icon of size %@; desired size is %@", __PRETTY_FUNCTION__, inObject, NSStringFromSize([[inObject userIcon] size]), NSStringFromSize(size));
NSLog(@"%s: Contact icon has representations %@", __PRETTY_FUNCTION__, [[inObject userIcon] representations]);

2007-11-03 20:16:48.034 Adium[23496:10b] +[AIUserIcons listUserIconForContact:size:]: Contact <AIListContact:109cf1a0 AIM.[deleted]> has icon of size {50, 37}; desired size is {50, 50}
2007-11-03 20:16:48.035 Adium[23496:10b] +[AIUserIcons listUserIconForContact:size:]: Contact icon has representations (
    NSCachedImageRep 0x109cff40 Size={16, 12} ColorSpace=NSCalibratedRGBColorSpace BPS=8 Pixels=16x12 Alpha=NO
)

So, it seems like NSImage's -size method is telling us vicious lies: this image (which appears pixelated in the CL) has one representation which is 16 x 12, yet it's claiming that its (the image's) size is 50 x 37.

Evan, I'm confused about how that fix in your test app applies to Adium. We don't set the size of the image in that method.

Replying to boredzo:

Evan, I'm confused about how that fix in your test app applies to Adium. We don't set the size of the image in that method.

We don't, indeed :)

My point above was that adding new calls to setSize:, at the locations in Adium equivalent to those suggested in the test app, seemed to be a potential fix. Equivalent would mean (1) the AIImageAdditions methods, directly applicable as seen in the test app, which applies to contact list drawing and (2) AITooltipUtilities just before the NSImageView's setImage: is called, which applies to tooltip display.

Also, a note on testing this bug (and the explicitly-dpi bug in #8012) within Adium: When setting a buddy icon, the surest way to make use of an actual file on disk for a buddy icon is to drag it into the image well (in personal prefs or the account prefs). If the icon is set via the image picker (double-clicking) via either drag and drop or its 'Choose' button, the image picker generally passes back a new, 72 dpi version of the image. I think I've seen it pass a higher-dpi version, but I don't have exact steps for trigerring that.

AIImageAdditions is far too late in the sequence for a fix. The bugged images are 16-pixel before they are sent to AIImageAdditions for scaling up: specifically, -[AIListObject userIcon], in those cases, returns a 16-pixel image, which AIUserIcons scales up to the Contact List's user-icon size for its cache.

The question now is where the 16-pixel images are coming from.

Progress:

2007-11-04 21:02:39.028 Adium[39533:817] -[ESUserIconHandlingPlugin updateListObject:keys:silent:]: inObject is <AIListContact:109cf9d0 AIM.[deleted]>; Cached image is NSImage 0x109d05d0 Size={50, 37} Reps=(
    NSBitmapImageRep 0x109d1110 Size={50, 37} ColorSpace=NSCalibratedRGBColorSpace BPS=8 BPP=32 Pixels=50x37 Alpha=NO Planar=NO Format=1
)
⋮
2007-11-04 21:13:21.613 Adium[39533:817] Somebody is requesting the KEY_USER_ICON array of <AIListContact:109cf9d0 AIM.saexcitebike>
(gdb) po array
<AIMutableOwnerArray: 109d0df0: (<ESUserIconHandlingPlugin: 0xde0cae0>:NSImage 0x109d05d0 Size={50, 37} Reps=(
    NSCachedImageRep 0x109c9280 Size={16, 12} ColorSpace=NSCalibratedRGBColorSpace BPS=8 Pixels=16x12 Alpha=NO
):1)>

Note the NSImages' pointers: It's the same image the whole way through. Apparently, in between, the NSImage creates its cached image rep — but it does so at 16-pixel size; in this case, 16×12.

I think I've located the source of the problem. We use -initByReferencingFile: to create the buddy icon images, and Apple's documentation of -[NSImage initByReferencingFile:] says:

If the cached version of the image uses less memory than the original image data, the original data is flushed and the cached image is used. (This can occur for images whose resolution is greater than 72 dpi.)

A 16-pixel image certainly does use less memory than a 50-pixel image.

So, where do we draw buddy icons at 16 pt? Certainly not my Contact List; it's set to 50 pt for testing this bug.

I'm not advocating any specific solution until we've located where the image is drawn at 16 pt.

Found it:

2007-11-04 22:06:26.441 Adium[39997:10b] +[AIUserIcons menuUserIconForObject:]: Scaling <AIListContact:11e049d0 AIM.[deleted]>'s icon for menu item
2007-11-04 22:06:26.443 Adium[39997:10b] -[NSImage(AIImageAdditions) imageByScalingForMenuItem]: Before: self is NSImage 0x11e05f40 Size={50, 50} Reps=(
    NSBitmapImageRep 0x111b6400 Size={50, 50} ColorSpace=NSCalibratedRGBColorSpace BPS=8 BPP=32 Pixels=50x50 Alpha=NO Planar=NO Format=1
)
2007-11-04 22:06:26.445 Adium[39997:10b] -[NSImage(AIImageAdditions) imageByScalingToSize:fraction:flipImage:proportionally:allowAnimation:]: Scaling from {50, 50} to {{0, 0}, {16, 16}}
2007-11-04 22:06:26.448 Adium[39997:10b] -[NSImage(AIImageAdditions) imageByScalingForMenuItem]: After:  self is NSImage 0x11e05f40 Size={50, 50} Reps=(
    NSCachedImageRep 0x11e29f10 Size={16, 16} ColorSpace=NSCalibratedRGBColorSpace BPS=8 Pixels=16x16 Alpha=NO
)

So the cause of the problem is that AIContactMenu (or something) is not being lazy about creating its menu items' icons. When we fix that, this problem should go away.

Of course, we may also want to see about getting NSImage to be a bit more tight-fisted about hanging onto that NSBitmapImageRep. And creating the cached image rep with the first rectangle the image gets drawn to, rather than the image's own size, is a bug in NSImage, IMO.

I don't think it'd be AIContactMenu since that doesn't create user icons on 1.1.x. Best bet would be the log viewer.

Does it only happen for contacts for whom a message window is opened? User icons are drawn for the menu items under the Window menu for open chats.

@evands: i am experiencing that in the message window the user icon is shown normally, it's just in the contact list looking pixelated (no matter if i have a chat window open or not).

Evan: No. I can reproduce this without even opening a chat window yet.

Replying to boredzo:

Found it: {{{ 2007-11-04 22:06:26.441 Adium[39997:10b] +[AIUserIcons menuUserIconForObject:]: Scaling <AIListContact:11e049d0 AIM.[deleted]>'s icon for menu item 2007-11-04 22:06:26.443 Adium[39997:10b] -[NSImage(AIImageAdditions) imageByScalingForMenuItem]: Before: self is NSImage 0x11e05f40 Size={50, 50} Reps=( NSBitmapImageRep 0x111b6400 Size={50, 50} ColorSpace=NSCalibratedRGBColorSpace BPS=8 BPP=32 Pixels=50x50 Alpha=NO Planar=NO Format=1 ) 2007-11-04 22:06:26.445 Adium[39997:10b] -[NSImage(AIImageAdditions) imageByScalingToSize:fraction:flipImage:proportionally:allowAnimation:]: Scaling from {50, 50} to {{0, 0}, {16, 16}} 2007-11-04 22:06:26.448 Adium[39997:10b] -[NSImage(AIImageAdditions) imageByScalingForMenuItem]: After: self is NSImage 0x11e05f40 Size={50, 50} Reps=( NSCachedImageRep 0x11e29f10 Size={16, 16} ColorSpace=NSCalibratedRGBColorSpace BPS=8 Pixels=16x16 Alpha=NO ) }}} So the cause of the problem is that AIContactMenu (or something) is not being lazy about creating its menu items' icons. When we fix that, this problem should go away. Of course, we may also want to see about getting NSImage to be a bit more tight-fisted about hanging onto that NSBitmapImageRep. And creating the cached image rep with the first rectangle the image gets drawn to, rather than the image's own size, is a bug in NSImage, IMO.

Are we sure that this a problem? The result of imageByScalingForMenuItem is stored in menuIconCache within AIUserIcons. The icons provided to the contact list are different NSImage instances, stored in a separate cache.

The possible points of cross-over:

• AIImageAdditions.m:217 where we could return a copy/autorelease of the same NSImage, which perhaps is a shallow rather than deep copy and therefore leads to modifications to one NSImage instance affecting the rrepresentations of another. (I don't know whether that is actuallyhow NSImage copy works or not). However, that should only happen if the new size and the original size are identical, and for the menu case we're requesting a size of 16x16, and logging shows that -[self size] should be returning 50x50 in your case above.
• AIImageAdditions.m:241 where we potentially copy an existing representation and add tht to the image rather than drawing into an appropriately sized image from scratch. That should only happen for animating images, in any case. Is that code path triggerred where you're seeing the bug?

Replying to evands:

The result of imageByScalingForMenuItem is stored in menuIconCache within AIUserIcons. The icons provided to the contact list are different NSImage instances, stored in a separate cache.

The result of -imageByScalingForMenuItem is not the problem.

-imageByScalingForMenuItem works by drawing the receiver into a new image (the result). The problem is that drawing the receiver changes the receiver; the receiver creates an NSCachedImageRep at the destination size (16×16, in our case), and throws away its original NSBitmapImageRep.

The receiver is the image that AIUserIcons owns, so when that image gets passed to the Contact List's contact cell for drawing, the image only contains that puny 16×16 cached rep, so it gets scaled up and looks pixelated.

Writing that, the solution is clear, now: Have -imageByScalingForMenuItem copy the receiver and draw the copy.

Wow - yeah, I misread the logging to refer to the return image, not the original image. That's wild! :)

Incidentally, I can't reproduce this bug on 1.1.4svn — only on trunk.

Another wrinkle: This only happens with GIF images, not PNG images (even if I convert them to indexed-color using pngnq). Strange.

AIContactMenu in trunk gets a lot of -imageByScalingForMenuItem for display, but from AIUserIcons. It's also used as the icon for chats (in the open chats list), as well as the Window menu version of it. I can't think of any other places other than the log viewer.

Another weird fact: It doesn't even work with all GIF files. I just converted one of my PNG files to GIF and it didn't do it.

Test app demonstrating the bug whereby drawing certain images into a smaller image result in those source images being stripped of their original image rep. Includes a friend's buddy icon (with permission) that reproduces the bug.

The CopyImage test app also shows that -[NSImage copy] is a deep copy: reps are copied, too. (That was the original point of the test app, as you might have guessed.)

The images i have cached are jpg. Which are also the same examples I uploaded

Failed workarounds:

• In -imageByScalingForMenuItem, send -recache to the original image after creating the resized version.
• Based on Apple's description of -initByReferencingFile: (“If you resize the image by less than 50%, the data is loaded in again from the file.”): After creating the resized version, try setting the original's size to something really small and then restoring it.
• In ESUserIconHandlingPlugin, load the image using -initWithContentsOfFile: instead of -initByReferencingFile:. (Suggested by David.)

This works:

• In -imageByScalingToSize:fraction:flipImage:proportionally:allowAnimation:, copy the image using -copy, then draw the copy.

(In [21554]) As of Mac OS X 10.5.0, drawing certain images into a smaller image results in a cached image rep being created at the destination image's size, rather than the source image's size, and the source image's original rep being thrown away. When we later went to draw the image at full size, the image was scaled up, resulting in a blocky mess.

The fix is simply to make a copy of the image using -copy and draw the copy. Lame, but it works, and it fixes #8181.

(In [21559]) Reverted r21558, which was definitely faster, but unfixed #8181. Refs #8181.

(In [21560]) This is an alternate fix suggested by David: use -initWithContentsOfFile: to create the NSImages for the buddy icons, and set their dataRetained property to YES. Then, we don't need to copy the image every time we scale it. I tested this fix and it works. Refs #8181.

• Performance upside: We don't copy the image every time we scale it. This should make changing the CL's buddy-icon size snappier (as snappy as it was before #8181 was originally fixed).
• Performance downside: Loading the cached images isn't lazy anymore. In reality, however, this costs us nothing: The user icons pretty much get drawn right away anyway, so they don't need to be lazily loaded.

I guess Apple doesn't consider this a bug - the description of setRetainsData explicitly describes the behavior.

1. I think we should change the way the cache works if this is needed; once we receive an icon from the contact over the wire, it sucks that we then keep both the new icon and the old cached file in memory.
2. Do we need to make a similar change to incoming images (vs. just to cached images) so that they don't lose quality as they are similarly scaled?

Setting to 1.2; change to 1.1.5 if this is merged (seems like it should be)

This broke resizing of animated GIFs. They're now really, really small.

An example of a resized animated GIF to 16x16 (in the contact list settings). It displays normally in the contact info pane, as well as in the 'Contacts' menu.

Sigh, this copy of trunk isn't latest; I didn't notice the compile failed. Ignore me.

This issue is still happening in r21596 if the display picture is downloaded from "Info" command it still pixelates. the images display correctly when downloaded normally but when brought on by using Info command it pixelates them still.

Odd. I have determined it still happens only after initially downloading icon using "info" command. Quitting and relaunching will use good icons instead of pixelated ones. So somehow when the icon is FIRST downloaded using info, the small cached version gets used until next launch where then the fixed method gets used.

Indeed. As I said above:

2. Do we need to make a similar change to incoming images (vs. just to cached images) so that they don't lose quality as they are similarly scaled?

r21646 pixelated icon after a fresh launch from an icon that was not just downloaded. :\

Replying to MysticalOS:

r21646 pixelated icon after a fresh launch from an icon that was not just downloaded. :\

Is that image in your Apple Address Book?

Ah my mistake. Forgot I dumped the entire caches folder so it re-downloaded everyones icon on relaunch, so it did pixelate the 5 icons on receiving. Another relaunch made all pixelizing go away. Usually i test for absent mindedness first. My mistake. heh. So we are still safe. It still only happens to just downloaded icons. But once cached appearance is correct.

(In [21652]) Call -[NSImage setDataRetained:YES] on incoming images from the server and from address book since these images may be scaled in multiple ways. This was done in [21560] for cached images; it needs to be done in these places, as well. Fixes #8181