Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
326 views
in Technique[技术] by (71.8m points)

r - Using Purrr Functions to iterate manual work

I have a set of code that I am using to scrape data by year. Each year has a large amount of data (545 MB-ish in the global environment as a tbl_df).

I have been successful in getting the code to run manually by season, but I'm trying to upgrade the code to iterate over multiple years.

My code initial takes dates stored in a tbl_df and eventually passes them as arguments to a function in another package.

Single Year Programming

library(tidyverse)
library(baseballr)
#> Registered S3 method overwritten by 'quantmod':
#>   method            from
#>   as.zoo.data.frame zoo

df <- 
tibble::tribble(
   ~year,   ~startDate,     ~endDate,
  "2021", "2021-04-01", "2021-10-03",
  "2020", "2020-07-23", "2020-09-27"
  )


# Single Year
df <- df %>% 
  filter(year == "2020")

# get regular season start and end dates for scrape
start_date <- df$startDate %>% lubridate::ymd()
end_date <- df$endDate %>% lubridate::ymd()

# create sequence of dates for scrape function
weekly_dates <- seq(start_date, end_date, by = "1 week") %>% as_tibble()

weekly_dates <- weekly_dates %>%
  rename(start_date = value) %>%
  mutate(end_date = start_date + 6) %>%
  mutate_all(as.character)

# Example of df  
weekly_dates
#> # A tibble: 10 x 2
#>    start_date end_date  
#>    <chr>      <chr>     
#>  1 2020-07-23 2020-07-29
#>  2 2020-07-30 2020-08-05
#>  3 2020-08-06 2020-08-12
#>  4 2020-08-13 2020-08-19
#>  5 2020-08-20 2020-08-26
#>  6 2020-08-27 2020-09-02
#>  7 2020-09-03 2020-09-09
#>  8 2020-09-10 2020-09-16
#>  9 2020-09-17 2020-09-23
#> 10 2020-09-24 2020-09-30

safe_statcast <- possibly(scrape_statcast_savant, otherwise = NA)

# Pass along referenced columns as the two arguments in the referenced function
return <- map2_dfr(weekly_dates$start_date, weekly_dates$end_date, ~ safe_statcast(.x,
                                                                                   .y,
                                                                                   player_type = "pitcher"
))


I'm looking for help with 3 things:

  1. Run some form of the code above but allowing multiple years of data with each year grouped together
  2. How would I save each years df as it's own *.Rds file? Something that uses the year column from above as it's own file name. I.E. saveRDS(return, here::here("data-raw", paste0(year, "_data.rds")))
  3. How would I mutate the same column across each year?

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...