IlegalAccessError throws when using browsermob with WebDriver 2.30.0 to capture n/w

[JaneLee]

Hi , lightbody,

I was so excited after reading a blog about how to capture n/w using WebDriver. One solution is to use browsermob-proxy. But unfortunately I met one issue during my practise project with latest firefox version. And it works not OK etither on IE8 or on Chrome browsers. Causing we need to cover mutliple browsers testing. Pls let me know how to solve this issue! Thx

---

Running environment:

Firefox 19.0.2
WebDriver-Server >= 2.27.0

---

Maven dependency configurations:

<dependencies>
    <dependency>
        <groupId>commons-lang</groupId>
        <artifactId>commons-lang</artifactId>
        <version>2.6</version>
    </dependency>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.8</version>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-server</artifactId>
        <version>2.30.0</version>
    </dependency>
    <dependency>
        <groupId>biz.neustar</groupId>
        <artifactId>browsermob-proxy</artifactId>
        <version>2.0-beta-7</version>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.seleniumhq.selenium</groupId>
                <artifactId>selenium-api</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

</dependencies>

---

My code using browsermob to capture n/w

@Test
// TODO: using Firefox failed
public void testFirefoxWithOBMP() throws Exception {
    // start the proxy
    ProxyServer server = new ProxyServer(5555);
    server.start();

    // get the Selenium proxy object
    Proxy proxy = server.seleniumProxy();

    // configure it as a desired capability
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability(CapabilityType.PROXY, proxy);

    // start the browser up
    System.setProperty("webdriver.firefox.bin", "D:\\Program Files\\Mozilla Firefox\\firefox.exe");
    WebDriver driver = new FirefoxDriver(capabilities);

    // create a new HAR with the label "innerhotsale"
    server.newHar("innerhotsale");

    // open re.taobao.com/search?
    driver.get(INNER_HOTSALE_URL);
    driver.manage().window().maximize();

    // get the HAR data
    Har har = server.getHar();

    System.out.println("============" + har.getLog().getEntries().size());

    System.out.println("END!!");

    server.stop();
}

---

Following are error logs:

INFO 03/13 07:12:09 o.b.p.j.h.HttpServer - Version Jetty/5.1.x
INFO 03/13 07:12:09 o.b.p.j.u.Container - Started HttpContext[/,/]
INFO 03/13 07:12:09 o.b.p.j.h.SocketLis~ - Started SocketListener on 0.0.0.0:5555
INFO 03/13 07:12:09 o.b.p.j.u.Container - Started org.browsermob.proxy.jetty.jetty.Server@12be1bd
WARN 03/13 07:12:23 o.b.p.j.h.HttpConne~ - GET http://re.taobao.com/search? HTTP/1.1
java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory
at org.browsermob.proxy.http.SimulatedSocketFactory.connectSocket(SimulatedSocketFactory.java:118)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:561)
at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:372)
at org.browsermob.proxy.http.BrowserMobHttpRequest.execute(BrowserMobHttpRequest.java:157)
at org.browsermob.proxy.BrowserMobProxyHandler.proxyPlainTextRequest(BrowserMobProxyHandler.java:270)
at org.browsermob.proxy.selenium.SeleniumProxyHandler.handle(SeleniumProxyHandler.java:185)
at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1509)
at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1461)
at org.browsermob.proxy.jetty.http.HttpServer.service(HttpServer.java:892)
at org.browsermob.proxy.jetty.http.HttpConnection.service(HttpConnection.java:815)
at org.browsermob.proxy.jetty.http.HttpConnection.handleNext(HttpConnection.java:981)
at org.browsermob.proxy.jetty.http.HttpConnection.handle(HttpConnection.java:832)
at org.browsermob.proxy.jetty.http.SocketListener.handleConnection(SocketListener.java:245)
at org.browsermob.proxy.jetty.util.ThreadedServer.handle(ThreadedServer.java:357)
at org.browsermob.proxy.jetty.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
WARN 03/13 07:12:24 o.b.p.j.h.HttpConne~ - GET http://re.taobao.com/favicon.ico HTTP/1.1
java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory
at org.browsermob.proxy.http.SimulatedSocketFactory.connectSocket(SimulatedSocketFactory.java:118)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:561)
at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:372)
at org.browsermob.proxy.http.BrowserMobHttpRequest.execute(BrowserMobHttpRequest.java:157)
at org.browsermob.proxy.BrowserMobProxyHandler.proxyPlainTextRequest(BrowserMobProxyHandler.java:270)
at org.browsermob.proxy.selenium.SeleniumProxyHandler.handle(SeleniumProxyHandler.java:185)
at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1509)
at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1461)
at org.browsermob.proxy.jetty.http.HttpServer.service(HttpServer.java:892)
at org.browsermob.proxy.jetty.http.HttpConnection.service(HttpConnection.java:815)
at org.browsermob.proxy.jetty.http.HttpConnection.handleNext(HttpConnection.java:981)
at org.browsermob.proxy.jetty.http.HttpConnection.handle(HttpConnection.java:832)
at org.browsermob.proxy.jetty.http.SocketListener.handleConnection(SocketListener.java:245)
at org.browsermob.proxy.jetty.util.ThreadedServer.handle(ThreadedServer.java:357)
at org.browsermob.proxy.jetty.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
============2
END!!
INFO 03/13 07:12:24 o.b.p.j.u.ThreadedS~ - Stopping Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=5555]
INFO 03/13 07:12:24 o.b.p.j.h.SocketLis~ - Stopped SocketListener on 0.0.0.0:5555
java.lang.NullPointerException
at com.etao.ad.p4p.hotsale.learn.testFirefox.afterClass(testFirefox.java:101)

YOURS,
Jane Li

[lightbody]

Jane,
I believe the problem is that you're using an old version of Java. We require Java 1.6 and it looks like you're running 1.5. Can you confirm?

[JaneLee]

Hi, lightbody,

My java version is "1.6.0_07".

The browsermob dependency of selenium-api version is 2.20.0. We tested when my project configured with following selenium-server version, browsermob works fine. But not OK with the version larger than 2.26.0. Pls help to confirm whether the error is related to it?

YOURS,
Jane

[lightbody]

Interesting. Do you have any unique policies/access rights on the JVM? How are you launching it?

java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory

On Mar 14, 2013, at 6:21 AM, Jane Li notifications@github.com wrote:

Hi, lightbody,

My java version is "1.6.0_07".

The browsermob dependency of selenium-api version is 2.20.0. We tested when my project configured with following selenium-server version, browsermob works fine. But not OK on the version larger than 2.26.0. pls help to confirm whether the error is related to it?

org.seleniumhq.selenium
selenium-server
2.20.0

YOURS,
Jane

—
Reply to this email directly or view it on GitHub.

[schwartzie]

I'm getting the exact same error using the 2.0-beta-7 standalone REST API. I'm using Java version 1.6.0_26 on a 64-bit Debian Squeeze VM.

[lightbody]

Can you boil it down to a bare bones test case?

On Mar 14, 2013, at 10:23 AM, schwartzie notifications@github.com wrote:

I'm getting the exact same error using the 2.0-beta-7 standalone REST API. I'm using Java version 1.6.0_26 on a 64-bit Debian Squeeze VM.

—
Reply to this email directly or view it on GitHub.

[schwartzie]

I didn't get any farther than a bare bones test case myself, so here's a complete rundown of what I did:

Launch proxy server:

$ cd browsermob-proxy-2.0-beta-7/bin/
$ sh browsermob-proxy -port 9090
INFO 03/14 18:14:12 o.b.p.Main           - Starting BrowserMob Proxy version 2.0-beta-7
INFO 03/14 18:14:12 o.e.j.u.log          - jetty-7.3.0.v20110203
INFO 03/14 18:14:12 o.e.j.u.log          - started o.e.j.s.ServletContextHandler{/,null}
INFO 03/14 18:14:12 o.e.j.u.log          - Started SelectChannelConnector@0.0.0.0:9090

Create proxy instance, and add page to HAR:

$ curl -X POST http://localhost:9090/proxy
{"port":9091}
$ curl -X PUT http://localhost:9090/proxy/9091/har

...results in the following output from the server:

INFO 03/14 18:14:31 o.b.p.j.h.HttpServer - Version Jetty/5.1.x
INFO 03/14 18:14:31 o.b.p.j.u.Container  - Started HttpContext[/,/]
INFO 03/14 18:14:31 o.b.p.j.h.SocketLis~ - Started SocketListener on 0.0.0.0:9091
INFO 03/14 18:14:31 o.b.p.j.u.Container  - Started org.browsermob.proxy.jetty.jetty.Server@785e8d7d

Make a request via the proxy:

$ curl -iL -x localhost:9091 http://google.com
HTTP/1.1 500 Internal Server Error
Date: Thu, 14 Mar 2013 18:14:58 GMT
Server: Jetty/5.1.x (Linux/2.6.32-5-xen-amd64 amd64 java/1.6.0_26
Connection: close

...results in the following output from the server:

WARN 03/14 18:14:59 o.b.p.j.h.HttpConne~ - GET http://google.com/ HTTP/1.1
java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory
    at org.browsermob.proxy.http.SimulatedSocketFactory.connectSocket(SimulatedSocketFactory.java:118)
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
    at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
    at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
    at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:561)
    at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:372)
    at org.browsermob.proxy.http.BrowserMobHttpRequest.execute(BrowserMobHttpRequest.java:157)
    at org.browsermob.proxy.BrowserMobProxyHandler.proxyPlainTextRequest(BrowserMobProxyHandler.java:270)
    at org.browsermob.proxy.selenium.SeleniumProxyHandler.handle(SeleniumProxyHandler.java:185)
    at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1509)
    at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1461)
    at org.browsermob.proxy.jetty.http.HttpServer.service(HttpServer.java:892)
    at org.browsermob.proxy.jetty.http.HttpConnection.service(HttpConnection.java:815)
    at org.browsermob.proxy.jetty.http.HttpConnection.handleNext(HttpConnection.java:981)
    at org.browsermob.proxy.jetty.http.HttpConnection.handle(HttpConnection.java:832)
    at org.browsermob.proxy.jetty.http.SocketListener.handleConnection(SocketListener.java:245)
    at org.browsermob.proxy.jetty.util.ThreadedServer.handle(ThreadedServer.java:357)
    at org.browsermob.proxy.jetty.util.ThreadPool$PoolThread.run(ThreadPool.java:534)

Retrieve HAR:

$ curl http://localhost:9090/proxy/9091/har
{"log":{"entries":[{"time":0,"request":{"method":"GET","headers":[],"queryString":[],"cookies":[],"bodySize":0,"url":"http://google.com/","httpVersion":"HTTP","headersSize":0},"response":{"content":{"size":0,"mimeType":""},"headers":[],"status":-999,"cookies":[],"bodySize":0,"httpVersion":"HTTP","headersSize":0,"statusText":"NO RESPONSE","redirectURL":""},"startedDateTime":"2013-03-14T18:14:58.926+0000","pageref":"Page 1","cache":{}}],"version":"1.1","browser":{"name":"cURL","version":"7.21.0"},"creator":{"name":"BrowserMob Proxy","version":"2.0"},"pages":[{"id":"Page 1","pageTimings":{},"startedDateTime":"2013-03-14T18:14:44.241+0000","title":""}]}}

Let me know if there's anything else I can provide that will help.

[JaneLee]

HI, lightbody,

To getting more clear with this issue, we tried to test on much more different scenarios.

Those are our testing result:

1. webdriver-server's version 2.30.0 vs browsermob-proxy'version 2.0-beta-6 =====> works fine
2. webdriver-server's version 2.30.0 vs browsermob-proxy'version 2.0-beta-7 =====> throws IlegalAccessError always.

Maybe this result can do some help. Pls help to check.

YOURS,
Jane

[schwartzie]

I just tried my test case with 2.0-beta-6, and it worked for me too.

[JaneLee]

will this issue be resolved at 2.0-beta-7 version?

[lightbody]

OK it looks like this all came from this comment:

e4c53e1#src/main/java/org/browsermob/proxy/http/SimulatedSocketFactory.java

I'm hoping that @rexhoffman can help us get it resolved. Rex?

[rexhoffman]

Hmm... I was using a mac os x version of java 6 when I wrote this, and didn't experience any security based exceptions/issues. I started a new gig since writing this however and don't have any tests set up around the code.

What os/jvms were used?

I have no need to support java 6 at this point. Just seemed a little sad to force 1.7 for one method. I believe the there were concerns regarding the time constraints of using other methods to determine the host. Though we never discussed caching.

Having done some work configuring policies/building hacks to prevent resource access during testing, this smells like a jvm doing security policy checking in the method.

Java security models are painful at best, and seems to diverge between different jvm providers, so my hack to work around getting access to non-public methods should probably be reverted -- since it has been proven to break with some jvms default policy configurations.

I'm not sure if another work-around would be accepted targeting some other method to calculate the host and potentially cache it.

Lesson learned: don't use reflection to grab non-public methods in the jvm -- they may get tightened up/be tighter in another jvm.

[lightbody]

@JaneLee and @schwartzie - can you get back to @rexhoffman with any info you have about the issue?

[JaneLee]

@rexhoffman @lightbody

My OS is window-xp; And using default jvm configration in Intellij IDEA.

[akosicki]

Just confirming the bug, java.lang.IllegalAccessError is being thrown when trying to use proxy.

Environment:

# browsermob-proxy version: 2.0-beta-7
$ uname -a
Linux aleksander-cuda 3.2.0-39-generic-pae #62-Ubuntu SMP Wed Feb 27 22:25:11 UTC 2013 i686 i686 i386 GNU/Linux
$ java -version
java version "1.6.0_43"
Java(TM) SE Runtime Environment (build 1.6.0_43-b01)
Java HotSpot(TM) Server VM (build 20.14-b01, mixed mode)

Steps to reproduce:

# as in tutotorial
$ sh browsermob-proxy -port 9090 &
...
$ curl -X POST http://localhost:9090/proxy
{"port":9091}
$ curl -x localhost:9091 http://google.com
java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory
    at org.browsermob.proxy.http.SimulatedSocketFactory.connectSocket(SimulatedSocketFactory.java:118)
...

[hyeung-trulia]

I got the same error on mac with JDK 1.7.