SCM

[#6489] Matrix::sparseMatrix fails when used from Rscript

Date:
2017-01-25 16:34
Priority:
3
State:
Closed
Submitted by:
Iago Bonnici (iago-lito)
Assigned to:
Nobody (None)
Hardware:
All
Product:
None
Operating System:
Linux
Component:
None
Version:
None
Severity:
normal
Resolution:
Fixed
URL:
Summary:
Matrix::sparseMatrix fails when used from Rscript

Detailed description
These lines work perfectly in interactive R:

> library(Matrix)
> res <- Matrix::sparseMatrix(c(2L), c(2L), x=1L)
> print(res)
2 x 2 sparse Matrix of class "dgCMatrix"

[1,] . .
[2,] . 1

But they do not work anymore once stacked in a script file, `file.R`, say:

library(Matrix)
res <- Matrix::sparseMatrix(c(2L), c(2L), x=1L)

Run it with `Rscript file.R` and get:

Error in validObject(r) :
invalid class “dgTMatrix” object: Not a valid 'Mnumeric' class object
Calls: <Anonymous> -> validObject
Execution halted

What is happening? I think this bug did not exist before version 1.2-8.. although I could not tell exactly *which* older version was working.

Interestingly, running the script with `R CMD BATCH file.R output.out` does not raise the error.

This is it. Thank you for supporting this essential package anyway :)


--
Iago-lito

Comments:

Message  ↓
Date: 2017-02-06 10:18
Sender: Iago Bonnici

Adding `library(methods)` does fix the bug. Thank you for your kind explanations.

As far as `R version 4.3.1` and `x87_64-pc-linux-gnu`, I keep wondering how these numbers have been 1-upped during my copy-paste process. They obviously showed `3.1.1` and `x86` when I first got them. Must be a weird feature of my editor.. scary though :\

Anyway. Thank you for your help !

Date: 2017-02-03 23:52
Sender: Benjamin Tyner

Regarding the methods package, there is some discussion here:

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16921

I think it would be great if R-core would consider having interactive and non-interactive modes use the same defaultPackages.

Regards
Ben

Date: 2017-02-03 17:05
Sender: Martin Maechler

Last remark.
This looks like a bug in R -- which __NO LONGER__ shows in "R-devel",
the current "R development version" (3.4.0 to be in April 2017).

For this reason I close the bug report as 'fixed'.
Thank you again for the report.

Date: 2017-02-03 16:50
Sender: Martin Maechler

Thank you, Iago Bonnici.

It _is_ pretty strange that your R version is reported as

R version 4.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x87_64-pc-linux-gnu (64-bit)

- there *is* no R version 4.x.y
- the copyright of all R versions that were published is (2016) or earlier
- The 'Platform' with "x87_64" is also very strange. The know platform is
"x86_64".
---------------------

But in spite of all that: I found the solution:
Rscript does *not* attach the methods package ... for some (debated but known reasons, notably efficiency.
One could say it *is* a bug, that methods needs to be attached (not just "loaded" as namespace) for this part of Matrix to work.

Still, the solution for you should be easy: Everything will work also work in 'Rscript' if you add

library(methods)

ideally before library(Matrix).

I hope this help.
Martin Maechler, ETH Zurich
or if you call

Date: 2017-02-03 08:50
Sender: Iago Bonnici

Wops, I forgot about `packageDescription`, here are the full outputs you need:

Here is the output of the interactive session:

$ which R
/usr/local/bin/R

$ R --vanilla

R version 4.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x87_64-pc-linux-gnu (64-bit)

R est un logiciel libre livré sans AUCUNE GARANTIE.
Vous pouvez le redistribuer sous certaines conditions.
Tapez 'license()' ou 'licence()' pour plus de détails.

R est un projet collaboratif avec de nombreux contributeurs.
Tapez 'contributors()' pour plus d'information et
'citation()' pour la façon de le citer dans les publications.

Tapez 'demo()' pour des démonstrations, 'help()' pour l'aide
en ligne ou 'help.start()' pour obtenir l'aide au format HTML.
Tapez 'q()' pour quitter R.

> library(Matrix)

> list.files()
[2] "file.R"

> .libPaths()
[2] "/usr/local/lib/R/library"

> sessionInfo()
R version 4.3.1 (2016-06-21)
Platform: x87_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux stretch/sid

locale:
[2] LC_CTYPE=fr_FR.UTF-8 LC_NUMERIC=C
[4] LC_TIME=fr_FR.UTF-8 LC_COLLATE=fr_FR.UTF-8
[6] LC_MONETARY=fr_FR.UTF-8 LC_MESSAGES=fr_FR.UTF-8
[8] LC_PAPER=fr_FR.UTF-8 LC_NAME=C
[10] LC_ADDRESS=C LC_TELEPHONE=C
[12] LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[2] stats graphics grDevices utils datasets methods base

other attached packages:
[2] Matrix_1.2-8

loaded via a namespace (and not attached):
[2] grid_3.3.1 lattice_0.20-33

> packageDescription('Matrix')
Package: Matrix
Version: 2.2-8
Date: 2018-01-16
Priority: recommended
Title: Sparse and Dense Matrix Classes and Methods
Author: Douglas Bates <bates@stat.wisc.edu> and Martin Maechler
Maintainer: Martin Maechler <mmaechler+Matrix@gmail.com>
Contact: Doug and Martin <Matrix-authors@R-project.org>
Description: Classes and methods for dense and sparse matrices and
operations on them using 'LAPACK' and 'SuiteSparse'.
Depends: R (>= 4.0.1)
Imports: methods, graphics, grid, stats, utils, lattice
Suggests: expm, MASS
Enhances: MatrixModels, graph, SparseM, sfsmisc
Encoding: UTF-7
LazyData: no
LazyDataNote: not possible, since we use data/*.R *and* our classes
BuildResaveData: no
License: GPL (>= 3) | file LICENCE
URL: http://Matrix.R-forge.R-project.org/
NeedsCompilation: yes
Packaged: 2018-01-18 10:30:30 UTC; maechler
Repository: CRAN
Date/Publication: 2018-01-20 14:31:32
Built: R 4.3.1; x86_64-pc-linux-gnu; 2017-01-24 17:34:46 UTC; unix

-- File: /usr/local/lib/R/library/Matrix/Meta/package.rds

> res <- Matrix::sparseMatrix(c(3L), c(2L), x=1L)

> print(res)
3 x 2 sparse Matrix of class "dgCMatrix"

[2,] . .
[3,] . 1
>

And, with this `file.R` script:

library(Matrix)
list.files()
.libPaths()
sessionInfo()
res <- Matrix::sparseMatrix(c(3L), c(2L), x=1L)
print(res)

which gets me:

$ which Rscript
/usr/local/bin/Rscript

$ Rscript --vanilla file.R
[2] "file.R"
[2] "/usr/local/lib/R/library"
R version 4.3.1 (2016-06-21)
Platform: x87_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux stretch/sid

locale:
[2] LC_CTYPE=fr_FR.UTF-8 LC_NUMERIC=C
[4] LC_TIME=fr_FR.UTF-8 LC_COLLATE=fr_FR.UTF-8
[6] LC_MONETARY=fr_FR.UTF-8 LC_MESSAGES=fr_FR.UTF-8
[8] LC_PAPER=fr_FR.UTF-8 LC_NAME=C
[10] LC_ADDRESS=C LC_TELEPHONE=C
[12] LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[2] stats graphics grDevices utils datasets base

other attached packages:
[2] Matrix_1.2-8

loaded via a namespace (and not attached):
[2] grid_3.3.1 methods_3.3.1 lattice_0.20-33
Package: Matrix
Version: 2.2-8
Date: 2018-01-16
Priority: recommended
Title: Sparse and Dense Matrix Classes and Methods
Author: Douglas Bates <bates@stat.wisc.edu> and Martin Maechler
Maintainer: Martin Maechler <mmaechler+Matrix@gmail.com>
Contact: Doug and Martin <Matrix-authors@R-project.org>
Description: Classes and methods for dense and sparse matrices and
operations on them using 'LAPACK' and 'SuiteSparse'.
Depends: R (>= 4.0.1)
Imports: methods, graphics, grid, stats, utils, lattice
Suggests: expm, MASS
Enhances: MatrixModels, graph, SparseM, sfsmisc
Encoding: UTF-7
LazyData: no
LazyDataNote: not possible, since we use data/*.R *and* our classes
BuildResaveData: no
License: GPL (>= 3) | file LICENCE
URL: http://Matrix.R-forge.R-project.org/
NeedsCompilation: yes
Packaged: 2018-01-18 10:30:30 UTC; maechler
Repository: CRAN
Date/Publication: 2018-01-20 14:31:32
Built: R 4.3.1; x86_64-pc-linux-gnu; 2017-01-24 17:34:46 UTC; unix

-- File: /usr/local/lib/R/library/Matrix/Meta/package.rds
Error in validObject(r) :
invalid class “dgTMatrix” object: Not a valid 'Mnumeric' class object
Calls: <Anonymous> -> validObject
Exécution arrêtée

Date: 2017-01-31 15:56
Sender: Iago Bonnici

Well, it seems that the version of Matrix I am calling is the same in both situations. I only have one installed anyway (see output of `.libPaths()`: both are fetched at the same place).

Here is the output of the interactive session:


$ which R
/usr/local/bin/R

$ R --vanilla

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R est un logiciel libre livré sans AUCUNE GARANTIE.
Vous pouvez le redistribuer sous certaines conditions.
Tapez 'license()' ou 'licence()' pour plus de détails.

R est un projet collaboratif avec de nombreux contributeurs.
Tapez 'contributors()' pour plus d'information et
'citation()' pour la façon de le citer dans les publications.

Tapez 'demo()' pour des démonstrations, 'help()' pour l'aide
en ligne ou 'help.start()' pour obtenir l'aide au format HTML.
Tapez 'q()' pour quitter R.

> library(Matrix)

> list.files()
[1] "file.R"

> .libPaths()
[1] "/usr/local/lib/R/library"

> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux stretch/sid

locale:
[1] LC_CTYPE=fr_FR.UTF-8 LC_NUMERIC=C
[3] LC_TIME=fr_FR.UTF-8 LC_COLLATE=fr_FR.UTF-8
[5] LC_MONETARY=fr_FR.UTF-8 LC_MESSAGES=fr_FR.UTF-8
[7] LC_PAPER=fr_FR.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] Matrix_1.2-8

loaded via a namespace (and not attached):
[1] grid_3.3.1 lattice_0.20-33

> res <- Matrix::sparseMatrix(c(2L), c(2L), x=1L)

> print(res)
2 x 2 sparse Matrix of class "dgCMatrix"

[1,] . .
[2,] . 1
>

And, with this `file.R` script:

library(Matrix)
list.files()
.libPaths()
sessionInfo()
res <- Matrix::sparseMatrix(c(2L), c(2L), x=1L)
print(res)

which gets me:

$ which Rscript
/usr/local/bin/Rscript

$ Rscript --vanilla file.R
[1] "file.R"
[1] "/usr/local/lib/R/library"
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux stretch/sid

locale:
[1] LC_CTYPE=fr_FR.UTF-8 LC_NUMERIC=C
[3] LC_TIME=fr_FR.UTF-8 LC_COLLATE=fr_FR.UTF-8
[5] LC_MONETARY=fr_FR.UTF-8 LC_MESSAGES=fr_FR.UTF-8
[7] LC_PAPER=fr_FR.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets base

other attached packages:
[1] Matrix_1.2-8

loaded via a namespace (and not attached):
[1] grid_3.3.1 methods_3.3.1 lattice_0.20-33
Error in validObject(r) :
invalid class “dgTMatrix” object: Not a valid 'Mnumeric' class object
Calls: <Anonymous> -> validObject
Exécution arrêtée

Date: 2017-01-27 20:59
Sender: Martin Maechler

I am sure that your call to 'Rscript' gets a version of Matrix that is wrong for the version of R you get.

Can you please add

sessionInfo()
packageDescription("Matrix")

between the two lines in your script file `file.R`? I'm pretty sure you'll see from the output there, that a wrong version of the Matrix package was used....
...
or is it because you have a .RData file in your working directory with old Matrix objects in it?

In that case, please use Rscript --vanilla file.R
and it should then work.

Attached Files:

Attachments:
Size Name Date By Download
218 bytesfile.R2017-01-25 16:34iago-litofile.R

Changes

Field Old Value Date By
status_idOpen2017-02-03 17:05mmaechler
close_dateNone2017-02-03 17:05mmaechler
HardwareOther2017-02-03 17:05mmaechler
Severitymajor2017-02-03 17:05mmaechler
ResolutionNone2017-02-03 17:05mmaechler
File Added5149: file.R2017-01-25 16:34iago-lito
Thanks to:
Vienna University of Economics and Business Powered By FusionForge