HttpWebRequest抓取不到HTML内容？

linautumn1006 · 发表于 2012-10-25 09:48:57

我的代码：
         string strResult = "";
         try
         {
            CookieContainer cookie = new CookieContainer();
            HttpWebRequest http = (HttpWebRequest)WebRequest.Create(url);
            http.AllowAutoRedirect = false ;
            http.KeepAlive = true;
            http.Method = "POST";
            http.Timeout = 5000;
            http.CookieContainer = cookie;
            HttpWebResponse response = (HttpWebResponse)http.GetResponse();
            Stream streamReceive = response.GetResponseStream();
            StreamReader streamReader = new StreamReader(streamReceive, Encoding .UTF8 );
            strResult = streamReader.ReadToEnd();
            streamReader.Close();
            streamReceive.Close();
            response.Close();
            http.Abort();
            return strResult;
}
catch{
...}

为什么我在访问“http://www.ria.ru/”网站的时候，得到的是
“<!DOCTYPE html SYSTEM "about:legacy-compat">
<html lang="ru"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><script src="/js/libs/jquery.min.js?"></script><script src="/js/libs/swfobject.js?"></script><script>var redirecturl = ''; var senddata = '{ "flash_support": ' + (swfobject.getFlashPlayerVersion().major > 0 ? 1 : 0) + ' }'; $.ajax({ type: 'POST', url: '/services/js_store', data: {action: "save", data: senddata}, success: function(data) { if (data.status == "OK") { if (redirecturl != '') { location.href = redirecturl; } else { location.reload(); } } }, dataType: "json" });</script><meta http-equiv="refresh" content="2;url=/"><noscript><meta http-equiv="refresh" content="0;url=/"></noscript></head><body></body></html>
”
而不是网页内容

站长苏飞 · 发表于 2012-10-25 10:22:11

你加上UserAgin试试
建议使用我的类来访问 http://www.sufeinet.com/thread-3-1-1.html
你的问题主要是这个界面不是直接显示内容，而是执行你得到的JS才来显示出来的

		自动登录	找回密码
密码			马上注册

[抓取] HttpWebRequest抓取不到HTML内容？