Re: data size limitations using gstat

["Edzer J. Pebesma" <edzer@pangea.Stanford.EDU>]

Sure, this helps.

No, gstat ignores the amount of white space.
It will be a bug in the progress report function
(i.e., only the % progress is in error, not 
necessarily the actual outcome).

The progress is reported by calculating the
number of point pairs done as a percentage of the
total # of point pairs. With 160000 records, the
total # of point pairs is 160000 * 160000 / 2,
which is 12800000000. This is bigger than what
can be stored in an unsigned int, so the counter
overflows. I will look into how to solve this bug,
but I don't consider it a a big one: the resulting
variogram will (probably) be right.

Your calculation evaluating 12800000000 point
pairs will take long anyway, so you may want to
sample from your data.
--
Edzer

"Soeren Hese (Tel.+49|30|67055-429)" wrote:
> 
> Thanks for the reply
> I am using a 3 column ascii file (x,y,value)
> generated from a 400x400 grid of multiview angle
> high res. digital image scanner data (forest).
> 
> The hardware is a Sun Ultra10 with solaris2.7 and
> 380 Mb RAM
> 
> gstat -v gives:
> 
> compiled on:          Nov 25 1999
> with libraries:       csf curses
> last modified on:     Tue Aug  3 13:10:04 CEST 1999
> gstat home page:      http://www.geog.uu.nl/gstat/
> questions, bugs etc.  mailto:gstat-info@geog.uu.nl
> 
> gstat: Sparc version 2.1.0 (August 1999)
> Copyright (C) 1992, 1999 Edzer J. Pebesma
> usage: gstat [options] [file]
> gstat: no command file
> 
> Starting gstat -i
> and using the ascii file with the default cutoff and width, direction :
> total and calculate what: semivariogram
> 
> the progress indicator gives different percent values incresing and
> decreasing and finally finishing with "ready" but no estimate file
> is dumped when I stop with Cntr-C (ready does not free the state of gstat
> again).
> 
> When using a cmd file as follows I receive a percent indication until 100%
> and then it adds a new line with 100% (at least 20 newlines) and
> finally proceeds
> with 101% - until ... I dont know -  at 2000 % I stopped it with Cntr-C .
> 
> cmd-file:
> 
> #
> # gstat command file, Sparc version 2.1.0 (August 1999)
> # Mon Jan 10 15:06:15 2000
> #
> data(fichte1): 'nd-fichte1-old.asc.clean', x=1, y=2, v=3;
> 
> method: semivariogram;
> #variogram(fichte1):'nd-fichte1-old.asc.clean';
> 
> set output = 'nd-fichte1-old.asc.clean.est';
> set logfile= 'gstat.log';
> #set debug=256;
> 
> When I use a datafile with 22000 lines it works very well and the
> datafiles are identical in terms of value format but the spacing is
> different - that might be a problem ( there are 4 spaces between every
> column in the big datafile instead of one space in the "22000 lines"-
> file).
> 
> Hope this helps somehow. Could be the case that I got it very
> wrong. Anyway if you can guess what I do wrong let me know - guess
> its something with spacing in the datafile.
> 
> All the best !
> 
> Soeren