Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
428 views
in Technique[技术] by (71.8m points)

html - get sourcecode after javascript execution with curl

Is it possible to get the html source code of a webpage with curl and the run a javascript interpreter over it, so i get the generated content?

The Page i need to get uses some encoded and genereated content in there so i want to first run the javascript to get the escaped and generated content... or do i need to regex the javascript and "compile" the javascript on my own? Like

curl <myurl> | perl -ne 'm/unescape((.*))/; print "$1"' | <now to something with that>

i known there is no javascript engine in curl but can i just call another script / programm to do the job?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

You can do it, but it's more involved than I think you realize. Neither curl nor wget have Javascript engines, so you'll need something that has one.

I would start by looking at PhantomJS.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...