[Antelope] Steim Compression Error

[Richard Baldwin]

Hi all,

A very minor point: the maximum difference allowed in Steim 2 
compression is 30 bits long. The actual range is -2^29 to +2^29-1, as 
stated in the SEED V2.4 (2007-10) manual, page 136.

Richard

Daniel Quinlan wrote:
> Hi Hank,
>
> The maximum difference allowed in steim 2 compression is 2^30, and 
> possibly the value of your
> first sample in this particular waveform, multiplied by your calib, is 
> greater than that.  The first
> difference saved in the compressed data is the difference between the 
> first sample and zero.  I suspect
> this is the reason you are getting an error message.
>
> Steim compression is intended to save integer data from a data logger, 
> not compress arbitrary data,
> and especially not floating point data.  So multiplying by calib and 
> then saving the data in integer
> format is a bit questionable.  You might instead save the calib 
> corrected data as floating point.
>
> -- danq
> On Nov 17, 2008, at 10:14 AM, Hank Ratzesberger wrote:
>
>>
>> Hi Kent,
>>
>> We have finally created the smallest example the re-creates the
>> compression error.  A zip file is available at:
>>
>> http://nees.ucsb.edu/pdf/problem_Example.zip
>>
>> with instructions in the readme for the checks we did
>> on the file and the results.
>>
>> We note that the saved file has one less sample, and the
>> first sample is bad.  Otherwise, the program is not much
>> more than the following:
>>
>> tr_loadcss()
>> tr_apply_calib()
>> tr_savewf()
>>
>> Regards,
>> Hank
>>
>>
>> On Nov 10, 2008, at 2:31 PM, Hank Ratzesberger wrote:
>>
>>> Thanks for your interest Kent.
>>>
>>> This is the program we use to apply the calib and
>>> copy the data to the new database.  (We use a very
>>> similar program which also calls rotate_to_standard.)
>>> It is called from a Perl script.
>>>
>>> Probably a good sign, but I cannot reproduce it in a
>>> minimal example.  We call this program repeatedly from
>>> a script that identifies all the true ZNE channels to
>>> apply the calib to the actual data.
>>>
>>> So, I think the issue is not the library, but using it
>>> under load of some kind.  Let me get back to you.
>>>
>>> Thanks,
>>> Hank
>>>
>>>
>>>
>>> <trcopy.c>
>>>
>>> Attached is the original file.
>>>
>>> <WLA_HLE_00_14179736.msd>
>>>
>>>
>>> Below is a trace showing the out of place first sample:
>>>
>>>
>>> <Picture 1.png>
>>>
>>>
>>>
>>> On Nov 8, 2008, at 4:26 PM, Dr. Kent Lindquist wrote:
>>>
>>>>
>>>> Hello Hank,
>>>>
>>>> Not to imply that I'm promising to debug and fix this, but the next 
>>>> step that would be nice is to make a very small, completely 
>>>> self-contained data-set; make a tiny little program that runs on it 
>>>> to reproduce the error; then explain precisely how the output of 
>>>> the script differs from what you expect. Also exactly where and 
>>>> from what would you 'drop the first data-point'? From the input, 
>>>> the processed trace structure, or the output? The reproducible 
>>>> example might help someone in the community see exactly what's 
>>>> occurring and help you resolve resolve it.
>>>>
>>>> Best regards,
>>>> Kent
>>>>
>>>>
>>>>
>>>> On Nov 8, 2008, at 6:07 AM, Hank Ratzesberger wrote:
>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> We have a program that calls the library routine trapply_calib 
>>>>> then saves
>>>>> the data to a new database.
>>>>>
>>>>> We recently noticed that some resulting traces have point "artifacts"
>>>>> that are way out of range and looked further to find this message
>>>>> in our log files:
>>>>>
>>>>> trcopy_osx: The difference -2147483648 between 0 and -2147483648 
>>>>> near sample #0 cannot be represented in a Steim level 2 compressed 
>>>>> record
>>>>>
>>>>> The program is running on an Intel Mac running Leopard.
>>>>> We are running some more tests to see if the problem does not
>>>>> occur on other platforms.
>>>>>
>>>>> We do not fill gaps.  The problem occurs on files that have no gaps.
>>>>>
>>>>> Any thoughts on why we get this error and what we might do about it?
>>>>> We could always drop the first data point -- if we knew how.
>>>>>
>>>>> Thanks,
>>>>> Hank
>>>>>
>>>>>
>>>>> Hank Ratzesberger
>>>>> NEES at UCSB
>>>>> Institute for Crustal Studies,
>>>>> University of California, Santa Barbara
>>>>> 805-893-8042
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Antelope mailing list
>>>>> Antelope at brtt.net
>>>>> http://brtt.net/mailman/listinfo/antelope_brtt.net
>>>>
>>>> -- 
>>>> Dr. Kent Lindquist                     kent at lindquistconsulting.com
>>>> Lindquist Consulting, Inc.
>>>> 59 College Rd. Suite #7
>>>> Fairbanks, AK 99701              Phone/FAX 907-457-2374
>>>>
>>>> http://www.lindquistconsulting.com
>>>>
>>>>
>>>> _______________________________________________
>>>> Antelope mailing list
>>>> Antelope at brtt.net
>>>> http://brtt.net/mailman/listinfo/antelope_brtt.net
>>>
>>> Hank Ratzesberger
>>> NEES at UCSB
>>> Institute for Crustal Studies,
>>> University of California, Santa Barbara
>>> 805-893-8042
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Antelope mailing list
>>> Antelope at brtt.net
>>> http://brtt.net/mailman/listinfo/antelope_brtt.net
>>
>> Hank Ratzesberger
>> NEES at UCSB
>> Institute for Crustal Studies,
>> University of California, Santa Barbara
>> 805-893-8042
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Antelope mailing list
>> Antelope at brtt.net
>> http://brtt.net/mailman/listinfo/antelope_brtt.net
>
>
> _______________________________________________
> Antelope mailing list
> Antelope at brtt.net
> http://brtt.net/mailman/listinfo/antelope_brtt.net