Forum: help

Place a full script in a loop, iterate through all files in folder [ Reply ]
By: Andrew Bannerman on 2017-08-28 03:17

I am want to run a full script over all files in a directory. So far I have pointed the code to my data directory. I then have a line of code to read the files names, in this case they are .txt files. Next, the for loop. Idea is to read each .txt file in the directory and loop along (f in 1:length(txt.files))

Inside the loop... it reads it as a data frame in each case... after we make new.df... I then enter my FULL SCRIPT which is about 289 lines which involves, working with the data, converting file formats, making calculations (other loops) and then printing out a plot which I save to the HD.

Issue I have is that the script is not running over all the files, even though I specified it to run the length of the txt.files.

This is what I setup:

# Set data dir
Data.dir <- "C:/R Projects/Data/"

# Set to read data dir, read all file names

txt.files <- file.path(dir("C:/R Projects/Data", pattern = ".txt", all.files = TRUE))

# Loop for script
for (f in 1:length(txt.files)) {
data.next <- paste(data.dir,txt.files[f],sep="")
nextfile <- read.table(data.next,header=TRUE, sep=",", stringsAsFactors=FALSE)
new.df <- data.frame(nextfile)

Full script is here... total of 289 lines
Script contains other loops within the script also. Loops that are making calculations.

}
If i follow the steps above this is my out put. All seems to be working:

> data.dir <- "C:/R Projects/Data/"
> txt.files <- file.path(dir("C:/R Projects/Data", pattern = ".txt", all.files = TRUE))
> length(txt.files)
[1] 9
> print(txt.files)
[1] "gdxj.30.min.txt" "gld_30min.txt" "spy_10m.txt" "spy_20min.txt" "spy_240min.txt" "spy_30_min.txt" "spy_5hr.txt"
[8] "spy_5min.txt" "spy_60min.txt"
> data.next <- paste(data.dir,txt.files[f],sep="")
> data.next
[1] "C:/R Projects/Data/gdxj.30.min.txt"
> nextfile <- read.table(data.next,header=TRUE, sep=",", stringsAsFactors=FALSE)
> new.df <- data.frame(nextfile)
> head(new.df)
Date Time Open High Low Close Up Down KeyRevDn KeyRevUp Volume
1 11/11/2009 09:00 104 108 101 102 30591 27618 0 0 0
2 11/11/2009 09:30 102 103 101 103 36514 19820 0 0 56334
3 11/11/2009 10:00 103 103 102 102 16624 25650 0 0 42274
4 11/11/2009 10:30 102 102 101 101 3709 12058 0 0 15767
5 11/11/2009 11:00 101 102 100 101 11319 9793 0 0 21112
6 11/11/2009 11:30 101 101 101 101 8742 9528 0 0 18270
>
It is reading all files in the directory and when i make a new.df... it prints the first .txt file data.

My issue is... once I run my full script, it does not loop through the rest of the .txt files. I have the for loop set to run for the length of txt.files. So not sure why its not looping. Does anyone have any ideas?