VCRProxy:记录PhantomJS阿贾克斯与VCR里面水豚电话里面、电话、PhantomJS、VCRProxy

2023-09-10 15:56:08 作者:我没抽风,是风在抽我

我已经做了一些这方面的研究,但没有发现任何解决方案。我有一个网站,在那里asynchron Ajax调用(使用JSONP)发到Facebook。我记录在Ruby侧VCR我所有的HTTP请求,所以我想它会很酷,使用此功能进行AJAX调用为好。

I already did some research in this field, but didn't find any solution. I have a site, where asynchron ajax calls are made to facebook (using JSONP). I'm recording all my HTTP requests on the Ruby side with VCR, so I thought it would be cool, to use this feature for AJAX calls as well.

于是我打了周围一点点,并想出了一个代理的尝试。我使用PhantomJS作为一个无头的浏览器和骚灵为内水豚的整合。鬼驱人现在配置为使用代理服务器是这样的:

So I played a little bit around, and came up with a proxy attempt. I'm using PhantomJS as a headless browser and poltergeist for the integration inside Capybara. Poltergeist is now configured to use a proxy like this:

  Capybara.register_driver :poltergeist_vcr do |app|
    options = {
      :phantomjs_options => [
        "--proxy=127.0.0.1:9100",
        "--proxy-type=http",
        "--ignore-ssl-errors=yes",
        "--web-security=no"
      ],
      :inspector => true
    }
    Capybara::Poltergeist::Driver.new(app, options)
  end
  Capybara.javascript_driver = :poltergeist_vcr

有关测试的目的,我写的基础上的WEBrick代理服务器,它集成VCR:

For testing purposes, I wrote a proxy server based on WEbrick, that integrates VCR:

require 'io/wait'
require 'webrick'
require 'webrick/httpproxy'

require 'rubygems'
require 'vcr'

module WEBrick
  class VCRProxyServer < HTTPProxyServer
    def service(*args)
      VCR.use_cassette('proxied') { super(*args) }
    end
  end
end

VCR.configure do |c|
  c.stub_with :webmock
  c.cassette_library_dir = '.'
  c.default_cassette_options = { :record => :new_episodes }
  c.ignore_localhost = true
end

IP   = '127.0.0.1'
PORT = 9100

reader, writer = IO.pipe

@pid = fork do
  reader.close
  $stderr = writer
  server = WEBrick::VCRProxyServer.new(:BindAddress => IP, :Port => PORT)
  trap('INT') { server.shutdown }
  server.start
end

raise 'VCR Proxy did not start in 10 seconds' unless reader.wait(10)

这可以很好地处理每一个本地主机通话,而他们得到良好记录。在HTML,JS和CSS文件通过VCR记录。然后我启用了 c.ignore_localhost = TRUE 选项,因为它是无用的(在我看来)来记录本地主机的电话。

This works well with every localhost call, and they get well recorded. The HTML, JS and CSS files are recorded by VCR. Then I enabled the c.ignore_localhost = true option, cause it's useless (in my opinion) to record localhost calls.

然后我又试了一次,但我必须搞清楚,属于该网页上的Ajax调用不被记录。更糟的是,他们并没有在测试里面工作了。

Then I tried again, but I had to figure out, that the AJAX calls that are made on the page aren't recorded. Even worse, they doesn't work inside the tests anymore.

因此​​,要开门见山,我的问题是:为什么所有调用中记录的本地主机JS文件,并JSONP调用外部ressources不?它不可能是JSONP的事情,因为它是一个正常的Ajax请求。或者是有内部phantomjs一个错误,那AJAX调用不是代理?如果是这样,我们如何解决这个问题?

So to come to the point, my question is: Why are all calls to JS files on the localhost recorded, and JSONP calls to external ressources not? It can't be the jsonP thing, cause it's a "normal" ajax request. Or is there a bug inside phantomjs, that AJAX calls aren't proxied? If so, how could we fix that?

如果它的运行,我想开始整合,并停止在

If it's running, I want to integrate the start and stop procedure inside

------- -------更新

我做了一些研究,并来到了以下几点:代理有一些问题通过HTTPS HTTPS电话呼叫和二进制数据。

I did some research and came to the following point: the proxy has some problems with HTTPS calls and binary data through HTTPS calls.

我启动服务器,并提出了一些卷曲电话:

I started the server, and made some curl calls:

curl --proxy 127.0.0.1:9100 https://m.xsw88.com/allimgs/daicuo/20230910/477.png

该电话被记录的,因为它应该。从代理请求和响应输出的是

This call gets recorded as it should. The request and response output from the proxy is

GET https://m.xsw88.com/allimgs/daicuo/20230910/477.png HTTP/1.1
User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
Host: d3jgo56a5b0my0.cloudfront.net
Accept: */*
Proxy-Connection: Keep-Alive

HTTP/1.1 200 OK 
Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-10-12)
Date: Tue, 20 Nov 2012 10:13:10 GMT
Content-Length: 0
Connection: Keep-Alive

不过,此调用不会被记录下来,一定是有问题的HTTPS:

But this call doesn't gets recorded, there must be some problem with HTTPS:

curl --proxy 127.0.0.1:9100 https://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png

头输出是:

CONNECT d3jgo56a5b0my0.cloudfront.net:443 HTTP/1.1
Host: d3jgo56a5b0my0.cloudfront.net:443
User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
Proxy-Connection: Keep-Alive

HTTP/1.1 200 OK 
Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-10-12)
Date: Tue, 20 Nov 2012 10:15:48 GMT
Content-Length: 0
Connection: close

所以,我想也许代理无法处理HTTPS,但它可以(只要我得到的卷曲调用后在控制台上的输出)。后来我想,也许VCR不能嘲笑HTTPS请求。但是使用这个脚本,VCR嘲笑了HTTPS请求,当我没有在代理中使用它:

So, I thought maybe the proxy can't handle HTTPS, but it can (as long as I'm getting the output on the console after the cURL call). Then I thought, maybe VCR can't mock HTTPS requests. But using this script, VCR mocks out HTTPS requests, when I don't use it inside the proxy:

require 'vcr'

VCR.configure do |c|
  c.hook_into :webmock
  c.cassette_library_dir = 'cassettes'
end

uri = URI("https://m.xsw88.com/allimgs/daicuo/20230910/478.png")

VCR.use_cassette('https', :record => :new_episodes) do
  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = true
  http.verify_mode = OpenSSL::SSL::VERIFY_NONE
  response = http.request_get(uri.path)
  puts response.body
end

那么,什么是问题? VCR处理HTTPS和代理处理HTTPS。他们为什么不一起玩?

So what is the problem? VCR handles HTTPS and the proxy handles HTTPS. Why they don't play together?

推荐答案

所以,我做了一些研究,我现在有一个工作VCR代理服务器,处理作为一个MITM访问代理服务器HTTPS调用(如果你禁用了一个非常基本的例子在您的客户端安全检查)。我会很高兴,如果有人能做出贡献,并帮我把这个东西的生活。

So I did some research and now I have a very basic example of a working VCR proxy server, that handles HTTPS calls as a MITM proxyserver (if you deactivate the security check in your client). I would be very happy if someone could contribute and help me to bring this thing to life.

下面是GitHub库: https://github.com/23tux/vcr_proxy

Here is the github repo: https://github.com/23tux/vcr_proxy