rvest - Scraping a complex HTML table into a data.frame in R

Question

I am trying to load wikipedia's data on US Supreme Court Justices into R:

library(rvest)

html = html("http://en.wikipedia.org/wiki/List_of_Justices_of_the_Supreme_Court_of_the_United_States")
judges = html_table(html_nodes(html, "table")[[2]])
head(judges[,2])

[1] "Wilson, JamesJames Wilson"       "Jay, JohnJohn Jay?"             
[3] "Cushing, WilliamWilliam Cushing" "Blair, JohnJohn Blair, Jr."     
[5] "Rutledge, JohnJohn Rutledge"     "Iredell, JamesJames Iredell"  

The problem is that the data is malformed. Rather than the name appearing how I see it in the actual HTML table ("James Wilson"), it is actually appearing twice, once as "Lastname, Firstname" and then once again as "Firstname Lastname".

The reason is that each actually contains an invisible :

<td style="text-align:left;" class="">
    <span style="display:none" class="">Wilson, James</span>
    <a href="/wiki/James_Wilson" title="James Wilson">James Wilson</a>
</td>

The same is also true for the columns with numeric data. I am guessing that this extra code is necessary for sorting the HTML table. However, I am unclear how to remove those spans when trying to create a data.frame from the table in R.

Answer 1

Maybe like this

library(XML)
library(rvest)
html = html("http://en.wikipedia.org/wiki/List_of_Justices_of_the_Supreme_Court_of_the_United_States")
judges = html_table(html_nodes(html, "table")[[2]])
head(judges[,2])
# [1] "Wilson, JamesJames Wilson"       "Jay, JohnJohn Jay?"              "Cushing, WilliamWilliam Cushing" "Blair, JohnJohn Blair, Jr."     
# [5] "Rutledge, JohnJohn Rutledge"     "Iredell, JamesJames Iredel

removeNodes(getNodeSet(html, "//table/tr/td[2]/span"))
judges = html_table(html_nodes(html, "table")[[2]])
head(judges[,2])
# [1] "James Wilson"    "John Jay?"       "William Cushing" "John Blair, Jr." "John Rutledge"   "James Iredell"