this sounds like a Selenium Core issue, but I couldn't find a field to mark it as such... its now labeled as BrowserBot-Mozilla.

Here's a patch. Now access to the entries list is synchronized. Additionally the code has been improved for better readability.

I'm running the selenium-server-standalone-2.0a4.jar on Windows XP controlling Firefox 3.6.3. I do get this error also sporadicly. Hope it can be fixed also in the 2.x release.

• Can we mark this back to selenium-rc or selenium-core? We see it with Safari, Chrome and Firefox.
• Also, it makes relying on this functionality for our tests rather difficult; would like to see it a MAJOR

Thanks!!

Stefan,

The codebase has changed, so your patch no longer applies cleanly. I appreciate you attaching one, but in the future please keep general code cleanups and actual bug fixes as separate patches. It's hard to figure out which changes are necessary and which are not.

If you're able to provide an updated patch against trunk, that'd be quite helpful. Otherwise, I'll try to extract the necessary portions sometime this week.

Hi Kevin,

I did as you suggested and dropped the 2.0.5 on top and we haven't seen this error since (knock on wood).

Thanks man!

Rob

Any word on whether this has already been fixed in 1.0.4 by some other means?

We're using a custom-spun jar that lets us use firefoxproxy on OS/X (the 2.0.5a jar does not work for us).

This is a Critical for me - using 1.0.3 I get ConcurrentModificationExceptions, with 2.0a5 I get no content in the XML document (nor JSON). I'm using this with selenium-webprofiler.

I would love to be able to get 1.0.4-SNAPSHOT if it has the fix, or even 1.0.3 source and apply the above patch, but I have no idea where that is, even looking at svn. There's no mention in svn of 1.0.3 afaik.

This patch may be applied to the trunk.

Patch SRC-836.patch is meant.

I synchronized access on all iteration. This should do the trick. Let me know if you still see the problem.

Hi Kevin,

Since you're in this code, there are two other bugs:

1) XML traffic capture does not escape ampersands in attributes. It mistakenly outputs:

<entry ... url="http://foo.com/bar?name=rob&last=cool"...>

Which when parsed looks like this:

entry[:url] = http://foo.com/bar?name=rob=cool

...because the '&' isn't escaped.

2) JSON traffic capture outputs invalid JSON. The keys should be inside of double-quotes, per the JSON spec.

[{statusCode:200}]

I will second J. Longman's comment - with the 2.x line, I also get no traffic captured with XML or JSON, so having all of these issues fixed in 1.x is becoming a show stopper for us

RE: JSON

To validate the output JSON, run it through jsonlint.com.

Currently being generated: [{statusCode: 200}] (invalid)

Should be generated: [{"statusCode": 200}]

Thanks Kevin!

Rob

Hi Robert,

Did you check the JSON output from trunk? That should have been fixed over a month ago. It should have been available in 2.0a5 as well.

As for new issues, please open separate issues. It'll just make it a lot easier to track things. We've also been moving over to Google Code for issue tracking. If you could open the XML one over there, it'd be much appreciated:

http://code.google.com/p/selenium/issues/list

Thanks,
Kevin

Silly me, filed a new bug:

http://jira.openqa.org/browse/SRC-840

Hi Kevin,

I haven't yet checked the 1.x trunk as I'm not sure where it lives =/

We're currently playing hot potato with our selenium-server-standalone.jar files. In some, *proxy browser modes won't work on OS / X, in some we get concurrent modification exception (sometimes) and in some we get no captureNetworkTraffic output as J. Longman points out.

We are sorely in need of a 1.0.4

Will you be at the meetup in Mountain View tonite?

Rob

1.x trunk is the same as 2.x trunk. Just check it out and run "./go selenium-server-standalone" to get the JAR you need. If you could verify, that'd help. Otherwise I could cut a 1.0.4 release that still ships with the problem.

I will be at SFSE tonight.

Hi Kevin,

Just built and installed on OS X 10.6.4 with FF 3.6.8.

The ConcurrentModificationException went away. Unfortunately, it's no longer returning network traffic

Using all types (XML, PLAIN, JSON), we get no traffic back.

Rob

It seems as though my results corroborate J. Longman's.

• On 1.0.3, ConcurrentModificationException
• On 1.0.4, no traffic returned.
• On 2.0.5, no traffic returned.

I'm on IRC at the moment, if you'd like me to svn up and rebuild for interactive verification.

Rob

Hi Kevin,

We had a bit of a look and inserted a few logging statements to understand what's happening.

Inside of ProxyHandler:

• Network traffic is captured inside of ProxyHandler::proxyPlainTextRequest(...).
• proxyPlainTextRequest() is only called by ProxyHandler::handle(...).
• handle() is never called.

...which means there's no way traffic can be captured.

Hope this helps!

Rob

Update!

• Traffic is captured with Google Chrome on OS/X
• Traffic is not captured with Firefox on OS/X (see my previous comment)

Unfortunately when using Google Chrome, we get an error from Selenium, which we don't get with Firefox (or Firefox via Sauce Labs).

Thu Aug 26 17:06:21 -0700 2010 selenium-client received failure from selenium server:
requested:
cmd=click
1=//a[@class='like_action']
sessionId=d8633f7b478840bfbd69a813c369bed6
received:
"ERROR: Command execution failure. Please search the forum at http://clearspace.openqa.org for error details from the log window. The error message is: Object #<a MouseEvent> has no method 'getPreventDefault'"
called from /Users/robertslifka/.rvm/rubies/ree-1.8.7-2010.02/lib/ruby/1.8/timeout.rb:62:in `timeout'
When I click on the share shade facebook like # features/step_definitions/str_steps.rb:64
ERROR: Command execution failure. Please search the forum at http://clearspace.openqa.org for error details from the log window. The error message is: Object #<a MouseEvent> has no method 'getPreventDefault' (Selenium::CommandError)

Can you check the proxy configuration from within Firefox the next time you debug? The browser shouldn't make any difference because the network data is captured at the proxy server level. In order for the data to be captured, however, the browser must use the Selenium server as the proxy. The Chrome launcher always sets up the proxy settings. The Firefox launcher should set up the proxy settings if the "captureNetworkTraffic" browser option is provided.

Hi Kevin,

Preferences > Advanced > Connection Settings:

Automatic proxy configuration URL:

file:/var/folders/rI/rI8bfk17E+OHA5AcSfjhiE+++TI/-Tmp-/customProfileDir42c85225226e461ebe4b81bdcdd79c51/proxy.pac

That file contains:

function FindProxyForURL(url, host) {
if (shExpMatch(host, 'local')) { return 'DIRECT'; }
if (shExpMatch(host, '*.local')) { return 'DIRECT'; }
if (shExpMatch(host, '169.254/16')) { return 'DIRECT'; }
if (shExpMatch(host, '*.169.254/16')) { return 'DIRECT'; }
if (shExpMatch(url, '/selenium-server/')) { return 'PROXY localhost:4444; DIRECT'; }
}

Note that I also get this warning on startup:

22:22:52.067 WARN - GET http://localhost:3001/selenium-server/core/scripts/user-extensions.js%5B42c85225226e461ebe4b81bdcdd79c51%5D HTTP/1.1
java.io.FileNotFoundException: (No such file or directory)

Hi Kevin,

I'm still working on wrapping my head around what's happening here.

If I modify DoNotUseProxyPac to output a file that looks like this:

function FindProxyForURL(url, host) {
  return 'PROXY localhost:4444';
}

Then it works! I can capture traffic with Firefox on OS/X.

Rob

Hi Kevin,

I think this line:

if (shExpMatch(url, '/selenium-server/')) { return 'PROXY localhost:4444; DIRECT'; }

...isn't working, because when I generated a proxy.pac that looked like this:

function FindProxyForURL(url, host) {
    if (shExpMatch(url, '/selenium-server/')) { return 'PROXY localhost:4444; DIRECT'; }
}

...it still didn't work. Either the matching logic failed or it fell back to DIRECT. It shouldn't have fallen back to DIRECT though, because the PROXY directive works in my one-line example above. My guess is that matcher is failing now?

HTH!

Rob

I filed a new issue at Google to track this proxy issue.

trunk (1.0.4?) proxy.pac generation with *firefoxproxy prevents request proxying
http://code.google.com/p/selenium/issues/detail?id=767

Please consider SRC-836 verified and closed.

Thanks!

Rob

Actually, the XML format is busted still. Bug in CaptureNetworkTrafficCommand:

sb.append("method=\"").append(json(entry.method)).append("\" ");

This generates:

method=""GET""

Should probably be "xml(entry.method)".

Here is a patch for Selenium 2.0 that solves all of the escaping problem with XML output. It uses org.apache.commons.lang.StringEscapeUtils