[Gretl-users] WG: append horizontal and vertical

Allin Cottrell cottrell at wfu.edu
Fri Sep 22 17:08:59 EDT 2017


On Fri, 22 Sep 2017, Allin Cottrell wrote:

> On Thu, 21 Sep 2017, Schaff, Frederik wrote:
>
>> unfortunately this works only for 2 but not 3 or more records... And I do 
>> not understand why.
>
> Frederik is referring to my script to do his data concatenation job using the 
> (recently tweaked) "append" command; see
> http://lists.wfu.edu/pipermail/gretl-users/2017-September/012690.html
> for details.
>
> I see what the problem is. First, there was a residual issue with the 
> "append" command, which I have now fixed in git. But beyond that, I was 
> assuming a greater regularity in Frederik's data files than actually obtains. 
> The following comment will mean little
> to anyone who hasn't tried working with Frederik's data, but here goes...
>
> My assumption was that the per-observation files
>
> stat_0-499_1.tsv
> stat_0-499_2.tsv
> stat_0-499_3.tsv
>
> (and other such tuples) contained observations (1,2,3) on a common set of 
> variables. Mostly that does seem to be the case -- and if it were always the 
> case my "append" recipe should work -- but it's not always the case. In 
> particular the file stat_0-499_3.tsv contains three variable identifiers that 
> are not found in any other file, namely
>
> Attendance_L_cv_I1
> Attendance_L_Skew_I1
> Attendance_L_Kurt_I1
>
> This discrepancy means the "append" (a fairly simple-minded command) will not 
> do the job [...]

Ah, but now I understand the source of the discrepancy, and it 
should be fixable, if not already fixed, by Frederik. The data files 
that he posted originally (the "1" and "2" files) contained the 
invalid identifiers:

Attendance_L-cv_I1
Attendance_L-Skew_I1
Attendance_L-Kurt_I1

These are invalid because "-" is an operator (minus) from gretl's 
point of view and you can't stick operators into identifiers. So 
gretl "auto-corrected" these names to

Attendance_Lcv_I1
Attendance_LSkew_I1
Attendance_LKurt_I1

(just dropping the offending character, as explained in the doc for 
gretl's fixname() function). Meanwhile, however, Frederik himself 
corrected these names to

Attendance_L_cv_I1
Attendance_L_Skew_I1
Attendance_L_Kurt_I1

(replacing '-' with '_') in the "3" file that he sent more recently.

So I presume that with current gretl git/snapshot my "append" recipe 
should work after all, provided one is working with a consistent set 
of Frederik's data files (either all auto-corrected in gretl's 
manner, or all manually corrected in an alternative but consistent 
manner).

Allin





More information about the Gretl-users mailing list