Skip to content

fread(yaml=TRUE) works when text=/input= is a single string with newlines #7938

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AnKiT-GaRG2 wants to merge 1 commit into
base: master
Choose a base branch
from
+9 −1
Open

fread(yaml=TRUE) works when text=/input= is a single string with newlines #7938

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions NEWS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@

21. GForce grouping is again faster when running single-threaded, e.g. after `setDTthreads(1)`, [#3330](https://github.com/Rdatatable/data.table/issues/3330). The batching used to parallelize `gsum()` and `gmean()` is skipped when only one thread is available. Also, `mean()` of an `integer` or `logical` column by group no longer coerces the whole column to `double` first, which made it slower than in v1.12.0 regardless of the number of threads. Thanks @jangorecki for the benchmarks, @mattdowle for the report, and @ben-schwen for the fix.

22. `fread(text=, yaml=TRUE)` and `fread(input=, yaml=TRUE)` now work when the data is a single character string containing newlines, rather than failing with `cannot open the connection`, [#7896](https://github.com/Rdatatable/data.table/issues/7896). Thanks to @Noskario for the report and @AnKiT-GaRG2 for the fix.

### Notes

1. {data.table} now depends on R 3.5.0 (2018).
Expand Down
3 changes: 2 additions & 1 deletion R/fread.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,8 @@ yaml=FALSE, tmpdir=tempdir(), tz="UTC")
.read_yaml_header = function(f, skip, verbose) {
# create connection to stream header lines from file:
# https://stackoverflow.com/questions/9871307
con = base::file(f, 'r')
# f can also be the data itself, a single string containing \n or \r from text= or input=, #7896
con = if (length(grep('\\n|\\r', f))) textConnection(f) else base::file(f, 'r')
# NB: close() won't be right if 'f' itself can be a file connection (#561)
on.exit(close(con))

Expand Down
5 changes: 5 additions & 0 deletions inst/tests/other.Rraw
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,11 @@ if (loaded[["yaml"]]) { # csvy; #1701. Was 2032-2033 in tests.Rraw, #5516
test(16.22, fread(f, skip = 'var1,', yaml = TRUE),
DT_yaml, warning = 'Combining a search.*YAML.*')

# yaml=TRUE with text= a single string containing newlines, #7896
txt = paste(readLines(f), collapse="\n")
test(16.23, fread(text=txt, yaml=TRUE), DT_yaml)
test(16.24, fread(txt, yaml=TRUE), DT_yaml)

# fwrite csvy: #3534
tmp = tempfile()
DT = data.table(a = 1:5, b = c(pi, 1:4), c = letters[1:5])
Expand Down
Loading