History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: SRC-376
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Haw-Bin Chai
Reporter: Haw-Bin Chai
Votes: 1
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Selenium Remote Control

POST Request with XML Content Filtered Out, Not Sent To Application Under Test By RC!

Created: 23/Oct/07 03:03 PM   Updated: 18/Nov/08 06:49 AM
Component/s: Server
Affects Version/s: 0.9.2
Fix Version/s: 1.0 beta 1

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown
Environment: Windows XP SP2, jre1.6.0u1, IE 6.0.2900, RC 0.9.3 snapshot revision 2012


 Description  « Hide
I originally posted about this here: http://forums.openqa.org/thread.jspa?threadID=10586&tstart=0

After doing more research, I've convinced myself this is a bug in the RC. Certain HTTP requests made by the page are not being passed back to the application under test by the RC; the failure to do so is not logged anywhere either. The result is that asynchronously loaded content is not loaded on the page - the request to the server never made it intact, so it makes sense that the response is not received either. This appears to affect IE more than Firefox; I have not observed any negative impact on Firefox.

I verified this by running IE with Paros Proxy as the "Proxy server for your LAN" (in the IE connection settings). When running directly through Paros, the asynchronously-loaded content appeared correctly, and I saw the requests were logged in the Paros window. However, when running through the RC to Paros (java -jar selenium-server.jar -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8081 -log server.log), I the same requests did NOT appear, and the content was NOT loaded in the browser.

Distinguishing characteristics of the failed requests seem to be 1) they are POST, and 2) they contain XML content. Is it possible that the RC doesn't like the format of these requests, and silently rejects them? If so, this is a problem! Here is what one representative request looks like (header, followed by content):


POST http://takin:18080/dsm/dynamicSearch.async HTTP/1.0
Accept: */*
Accept-Language: en-us
Referer: http://takin:18080/dsm/gotoHome.do
Pragma: no-cache
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30) Paros/3.2.13
Host: takin:18080
Content-Length: 19
Proxy-Connection: Keep-Alive
Cookie: JSESSIONID=8F488E555EBD40D0647A2332035DC986


<request></request>

 All   Comments   Work Log   Change History      Sort Order:
Haw-Bin Chai - 29/Oct/07 11:08 PM
It appears that around line 360-ish in ProxyHandler.java, the URLConnection output stream isn't being closed after the request is written to it:


            if (hasContent) {
                connection.setDoOutput(true);
                IO.copy(in, connection.getOutputStream());
            }


In examples I'm seeing on the web, the stream is closed after writes, sometimes with a flush() too. Unless IO.copy() automatically closes the stream after writing to it ... where the heck is IO.copy from again???

Also, same thing later in the file with ModifiedIO.copy(); the output stream isn't being closed.

Haw-Bin Chai - 30/Oct/07 01:41 PM
Ok, my previous comment was a red herring. I found the root cause.

ProxyHandler.java uses some logic to determine whether a given request has content:


    if (HttpFields.__ContentType.equals(hdr) && !"GET".equals(request.getMethod()))


If the request does NOT contain the "Content-Type" header, the RC assumes (perhaps erroneously) that the request does NOT contain content. What's the workaround? Well, you could simply make sure the request has the "Content-Type" header ... using setRequestHeader() for requests created in javascript, for example. We could also enhance the logic in ProxyHandler.java to consider the "Content-Length" header too, which might be present even if "Content-Type" isn't. This is the case for certain requests begin sent by IE. Here's a patch for that:


Index: ProxyHandler.java
===================================================================
--- ProxyHandler.java (revision 2068)
+++ ProxyHandler.java (working copy)
@@ -304,6 +304,7 @@

         // copy headers
         boolean xForwardedFor = false;
+ boolean isGet = "GET".equals(request.getMethod());
         boolean hasContent = false;
         Enumeration enm = request.getFieldNames();
         while (enm.hasMoreElements()) {
@@ -315,7 +316,7 @@
             if (connectionHdr != null && connectionHdr.indexOf(hdr) >= 0)
                 continue;

- if (HttpFields.__ContentType.equals(hdr) && !"GET".equals(request.getMethod()))
+ if (!isGet && HttpFields.__ContentType.equals(hdr))
                 hasContent = true;

             Enumeration vals = request.getFieldValues(hdr);
@@ -326,6 +327,9 @@
                     if ("Referer".equals(hdr) && (-1 != val.indexOf("/selenium-server/"))) {
                         continue;
                     }
+ if (!isGet && HttpFields.__ContentLength.equals(hdr) && Integer.parseInt(val) > 0) {
+ hasContent = true;
+ }

                     connection.addRequestProperty(hdr, val);
                     xForwardedFor |= HttpFields.__XForwardedFor.equalsIgnoreCase(hdr);



Dan Fabulich - 30/Oct/07 03:29 PM
That makes sense. You may note that I touched this line in revision 1939 to fix SRC-132. In hindsight, I think it's pretty weird that the ProxyHandler used ContentType to determine whether the request was a POST request or not.

Feel free to check this in...

Haw-Bin Chai - 30/Oct/07 04:49 PM
Commit 2070 includes the patch.