I wanted to build an awesome place for people to discuss module specific issues, but I don't have any more time for this, and there are much better places to discuss Perl-related issues. I'd recommend asking your question on Stack Overflow or on Perl Monks.
If you are looking for a Perl tutorial or Perl-related news, I hope these links will serve you well.
Posted on 2008-11-01 01:05:45-07 by gmikol
Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Hi--

I've have a bunch of Canon CRW files from a PowerShot S70 with 640x480 JPGFromRaw images. I've generated new, larger JPG's that I want to embed into the CRW files. I'm having trouble with it, though, and the Canon viewer, ZoomBrowser (ZB), isn't recognizing the new file fully.

The commands I'm using (in Windows) are:
exiftool -overwrite_original "-JPGFromRaw<=new_preview.jpg" TEST.CRW exiftool -overwrite_original "-ThumbnailImage<=new_thumb.jpg" TEST.CRW

Bigger problem is that my results are variable depending on the order of the steps:

1) If I just write the ThumbnailImage, everything works fine
2) If I just write the JPGFromRaw, ZB recognizes the file and can convert the RAW, but it does not use the JPGFromRaw image for viewing
3) Same results as #2 if I write the JPGFromRaw and then the ThumbnailImage
4) If I write the ThumbnailImage and then the JPGFromRaw, then the file is corrupted, ZB doesn't recognize it, and it can't convert the RAW.

I tried using the -F option in case it was a MakerNotes offset, but the results are the same.

I really don't know how to troubleshoot any further myself, so I'm hoping for some collective help.

I'm using the latest release (7.51)

Thanks--

Greg
Direct Responses: 9177 | Write a response
Posted on 2008-11-01 06:35:43-07 by bogdan in response to 9173
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Hi Greg,

I don't have any CRW file by hand, but if I recall correctly, then CRW files don't contain embedded thumbnails -thumbnails are saved into separate THW files.

Bogdan
Direct Responses: 9178 | 9180 | Write a response
Posted on 2008-11-01 07:15:52-07 by victor220 in response to 9177
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Professional golf gives you the opportunity to play a game you love, to earn big money doing it, and to possibly write your name in the pages of golfing history. As a professional golfer your place of business will be the golf course and the practice range. You won't be stuck behind a desk or in an office cubicle, and your job description - well, you get to write that yourself. You will share the game with other players who love it as well. As you travel from tournament to tournament, your competitors will become your friends. Together you will see the world and share a game with a legendary heritage So choosing a club which is fit you is very important. I suggest you could have a look at the www.golforderonline.com
Direct Responses: Write a response
Posted on 2008-11-01 14:42:15-07 by exiftool in response to 9177
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Thanks for reporting this problem.

I have the Canon utilities here so I will test this out. It is understandable that ZoomBrowser has problems if the Jpgfromraw is the wrong size, but there must be some quirk about the ordering of data within the file that it causing problems when writing the thumbnailimage first.

I will get to this as soon as I can, but I am away this weekend so give me a few days.

(Bogdan: Some powershot models do store a thumbnail inside the CRW -- EOS models use a separate THM file.)

- Phil
Direct Responses: 9183 | Write a response
Posted on 2008-11-01 15:41:19-07 by gmikol in response to 9180
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Actually, my PowerShot (S70) does both. It writes a separate THM and embeds the ThumbnailImage in the CRW. Don't know why it does that, but I figured all this out while I was trying to figure out what utilities use the embedded, and which use the THM.

You say "...if the JPGFromRaw is the wrong size...". I got the impression from the documentation that ExifTool was capable of resizing the ThumbnailImage and JPGFromRaw fields. I've stepped back and tried to use a 640x480 image instead of a higher-res one, and still got the same problems. This way, I shouldn't have to worry about the RawJpgInfo tags.

Let me know if you need sample files...

--Greg
Direct Responses: 9187 | Write a response
Posted on 2008-11-02 17:57:29-08 by exiftool in response to 9183
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Hi Greg,

Yes, exiftool can resize the fields, but my concern is that the zoombrowser may be expecting an image of certain dimensions (if the software is very inflexible). This is possible, but I don't know how likely it is.

An S70 sample would be useful. (You can email it to me at phil at owl.phy.queensu.ca). I have tested this with S30, S40, S50 and G3 CRW images (I don't have an S70 CRW handy) using Canon ImageBrowser 3.0 (the Mac equivalent of zoombrowser), but it doesn't have a problem with any of the images. I have tried deleting both thumbnailimage and jpgfromraw then adding them back again in both orders and it doesn't complain about anything. So it is either something different with S70 images, something different about zoombrowser or something different that you are doing. Do you have the same problem if you extract the thumbnailimage and jpgfromraw from an original CRW image then put the same images back into the CRW? ie)

exiftool -thumbnailimage -b -w thumb.jpg image.crw exiftool -jpgfromraw -b -w jfr.jpg image.crw exiftool -thumbnailimage= -jpgfromraw= image.crw exiftool -thumbnailimage"<=thumb.jpg" image.crw exiftool -jpgfromraw"<=jfr.jpg" image.crw

If this doesn't cause problems for you, please send me a set of commands that does.

Also, what version of zoombrowser are you using? I will see if I can load the same version onto the PC to duplicate the exact steps that are causing you problems.

- Phil
Direct Responses: 9196 | Write a response
Posted on 2008-11-03 08:51:42-08 by gmikol in response to 9187
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Phil--

First off, I'm using ZoomBrowser 6.11 under WinXP SP3. ActivePerl 5.10.0 Build 1003, ExifTool 7.51

Thanks for your help...I don't have any trouble with the set of commands you provided. The problem comes from trying to write a different file into the JPGFromRaw tag.

exiftool -thumbnailimage"<=new_thumb.jpg" image.crw
Seems to work fine by itself.

exiftool -jpgfromraw"<=new_jfr.jpg" image.crw
Breaks the display of the preview image in ZB. It looks like it takes the thumb and up-rezzes it to the correct pixel size instead. But it will still process the RAW.

exiftool -jpgfromraw"<=new_jfr.jpg" image.crw exiftool -jpgfromraw"<=jfr.jpg" image.crw
Works fine as well, but not very useful.

exiftool -thumbnailimage"<=new_thumb.jpg" image.crw exiftool -jpgfromraw"<=new_jfr.jpg" image.crw exiftool -jpgfromraw"<=jfr.jpg" image.crw
Breaks the file to the point that the Canon software won't convert the RAW. Same result if I clear the JPGFromRaw tag before writing the JFR image back. But it works fine if, as the 4th command, I write the original thumbnail back.

I've sent you a ZIP file with all the necessary pieces you can use to test this for yourself. The new_jfr.jpg file is 640x480 for now, the same size as the original jfr.jpg. Ultimately, I'd like to embed a bigger JFR, but I'll worry about that when (if) this gets working. Also, is there anyone out there who is successfully writing new JPGFromRaw images into your CRW files and they still work in ZoomBrowser? --Greg
Direct Responses: 9198 | Write a response
Posted on 2008-11-03 11:22:32-08 by exiftool in response to 9196
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Hi Greg,

Thanks for the samples. I will test this in the PC when I get home tonight, but I have an idea that you can test: Try deleting the JFIF information from your new thumb and jfr images (with "-jfif:all=") before writing them to the CRW. Just looking at the files, this difference jumps out at me and I'm wondering if it could cause problems somehow for the Canon software.

- Phil
Direct Responses: 9202 | Write a response
Posted on 2008-11-03 16:10:41-08 by gmikol in response to 9198
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser

I tried this, and got the same results. The files I was originally playing around with last week when I found these problems didn't have any JFIF tags. But those files had a different YCbCr Subsampling parameter than the Canon originals. So I used an IrfanView plugin that allows you to specify this parameter in the JPG compression, but it didn't make any difference.

--Greg
Direct Responses: 9211 | Write a response
Posted on 2008-11-04 00:18:39-08 by exiftool in response to 9202
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
yes, i thought about YCbCr too, but noticed it was the same.

I fired up the PC tonight, hoping my zoombrowser would have the same problem, but it didn't. I was using zoombrowser EX 4.1.1, which is from 2003. After following the same steps as you it still appeared to display the jpgfromraw, but it wouldn't convert the raw image (even for the original) -- it must be too old for the S70. So my next step is to update my zoombrowser, but this will have to wait until tomorrow. - Phil
Direct Responses: 9215 | Write a response
Posted on 2008-11-04 23:37:29-08 by exiftool in response to 9211
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Well, zoombrowser 6.1.1 wouldn't install for me (it gave an extract error), but the good news is imagebrowser 6.1.1 installed OK and gave me the same symptoms that you were observing.

So I have been able to reproduce the problem and it is solved. This was partly a bug in exiftool and partly a difference with the new utilities. The problem was with data alignment, and the problem only manifested itself for images with an odd number of bytes.

Thanks for the bug report, for the samples to allow me to reproduce this, and for your patience.

I have just released exiftool 7.52 which should solve this for you.

- Phil
Direct Responses: 9217 | Write a response
Posted on 2008-11-05 04:34:31-08 by gmikol in response to 9215
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Phil--

Thank you for your response and attention to this issue. I just downloaded the new version, installed it and tested it out on the same set of images I sent you.

The issue with the CRW image file being completely corrupted is resolved. It seems that no matter what order I write tags in, it's still convertable by Raw Image Task. So that's good.

Perhaps the other issue can't be resolved, but when I write the new_jfr.jpg to JPGFromRaw, it still doesn't show up in ZB correctly. It's still a highly pixelated version of the thumbnail, instead of the obvious new JPGFromRaw image. Is this working fine on your Mac version (ImageBrowser)? Did you confirm this part of the functionality?

--Greg
Direct Responses: 9219 | Write a response
Posted on 2008-11-05 11:33:53-08 by exiftool in response to 9217
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Hi Greg,

I did confirm that this worked for the Mac version (I even generated a jpgFromRaw that was a different image from the thumbnail for some of my tests to make it obvious which was displayed).

I'll have another go at installing zoombrowser to see if I can figure this one out. Did you confirm that the jpgFromRaw is displayed if you put the original one back into the file?

- Phil
Direct Responses: 9232 | Write a response
Posted on 2008-11-06 06:35:57-08 by gmikol in response to 9219
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Yeah...I can take it all apart and put it back together the way I found it. And it works fine, which is a definite improvement, but I can't embed a *new* JPGFromRaw into the CRW and have it display properly in ZoomBrowser.

--Greg
Direct Responses: 9234 | Write a response
Posted on 2008-11-06 11:45:58-08 by exiftool in response to 9232
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
OK, thanks Greg. Any testing you can do will help. We should be able to figure this one out. I've re-downloaded the ZB 6.1.1 update but it still won't install on my PC running XP service pack2. My next step is to see if I can find another PC to install it on.

I repeated the tests with ImageBrowser, and it worked flawlessly, displaying whatever JpgFromRaw I write without a hiccup (although I admit that I have only tried 640x480 images so far). So the is a ZB-specific quirk.

- Phil
Direct Responses: 9240 | Write a response
Posted on 2008-11-06 19:27:48-08 by gmikol in response to 9234
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Sure thing, Phil...I'm happy to do any testing. Just let me know what you need...

--Greg
Direct Responses: 9262 | Write a response
Posted on 2008-11-07 23:16:15-08 by exiftool in response to 9240
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Well, after a couple more wasted hours trying to install the zoombrowser update, I finally got it to work, so I have been able to try a few things myself. The symptoms I observe are different than you: I can't get ZB to display either a thumbnail or a jpgfromraw other than the original. I wonder if we are running a slightly different version of ZB. I'm running 6.1.1.21. Also, I am running Service pack 2, but I don't see how that could affect things.

But after a number of tests, I have narrowed down the possible causes for not displaying the embedded images:

- it isn't the YCbCrSubSampling (but you knew that already)

- it isn't any of the metadata (ie. JFIF)

- it isn't the image dimensions

- it isn't the image file size (I tried adding junk to the end of both the original and your new images to make them the same length, but the Canon ones still worked while the new ones didn't)

- it isn't the image encoding process (all baseline DCT, Huffman)

I could only think of one thing that was left:

- it could be that the quantization tables are different

So I did some specific tests and confirmed that this is the reason. If I copy the quantization tables (DQT segment) and huffman tables (DHT segment) from the Canon image and use them in the new image, then ZB will display this embedded image. Very odd. (Copying either the DQT or the DHT alone doesn't work, they both must be copied.) The only reason I could imagine why this would happen is if Canon intentionally checks for Canon-specific tables and disables display of other embedded images.

Very crazy indeed, but those are the facts.

Unfortunately I know if no utilities that will allow you to use a specific set of quantization tables. I had the bright idea of using ZB to rewrite the JPEG before embedding it, but sadly that didn't work.

So it looks like there is no simple solution to the problem, short of finding or writing a utility to use a specific set of quantization tables in the JPEG transform.

- Phil
Direct Responses: 9263 | Write a response
Posted on 2008-11-08 02:49:21-08 by gmikol in response to 9262
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Well, thanks for all your help, Phil.

For everyone who reads this in the future. JpgFromRaw images embedded with ExifTool v7.52 will display correctly in ZoomBrowser 5.0 and 5.5 (and presumably 5.7, which I didn't test). They do not display correctly in 6.01 and 6.11.

It's a shame Canon changed the way they're parsing the CRW files, if I thought I could get Canon to do anything about it, I would beg and whine and cry piteously until they did. But I can't...so I won't.

Thanks again--

--Greg
Direct Responses: 9266 | Write a response
Posted on 2008-11-08 14:53:27-08 by bogdan in response to 9263
Re: Writing JPGFromRAW (CRW) breaks Canon ZoomBrowser
Hi,

I didn't dig deeper into this, but... As far I recall, for ZB v6.xx, ZoomBrowser started to use .NET library. Maybe this is somehow related with issue you have. It's just a though.

Greetings,
Bogdan
Direct Responses: Write a response