I wanted to build an awesome place for people to discuss module specific issues, but I don't have any more time for this, and there are much better places to discuss Perl-related issues. I'd recommend asking your question on Stack Overflow or on Perl Monks.
If you are looking for a Perl tutorial or Perl-related news, I hope these links will serve you well.
WWW-Mechanize
Posted on 2009-08-29 10:38:10-07 by greghauptmann
Why won't this Mechanize HTTP POST work? (session timeout)
Hi, A question for the web/http guru's out there if I may (who understand cookies, sessions ids, etc).

BACKGROUND:
==========
I'm writing some code (using Mechanize) to automate logging into my bank account to check the balance. The first step after I successfully log and click on a button am getting "You Session Has Ended"! I've simulated using the bank account on my browser manually without Javascript enabled so this shouldn't be an issue (I note that someone posted Mechanize doesn't support javascript). I'm wondering what it could possibly be & what tools/approach I could you to prove the request from Mechanize exactly matches the successful post that occurs when I log in through my browser.

QUESTIONS:
==========
Q1 - Based on the below can someone see why I am getting a "You Session Has Ended"? (e.g. it it something to do with SessionId/cookies/full post parameters being in place). Some ideas, however not sure if they are significant:

(a) I had to do some work arounds as Mechanize didn't seem to be creating a valid URI re the first part of the URL, hence I hand modified it. I think it should be OK? It the line in the code "continue_form.action = "https://banking2.anz.com/IBAU/" + continue_form.action"

(b) The form itself (see below) actually doesn't have a SUBMIT button but has a "INPUT TYPE="Image" tag. Mechanize doesn't seem to pick this up, but when I do mechanize "submit" via the code "page = agent.submit(continue_form)" it still seems to initiate a POST to the server, albeit the response that comes back says the session has expired. I note there are some missing parameters in the browser trace versus the Mechanize trace (see below), but would this cause a "session is no longer valid"? (I tried to manually add fields to compensate into the form via the code "continue_form.add_field!("Action.RetUser.SignonOK","Proceed to Internet Banking")" however it didn't seem to trigger them to be included)

Q2 - What tool can I use on both the Browser & my Mechanize code to compare the exact POSTs and see where the differences are? I can do Live HTTP Headers in firefox for the browser test (which works), but in Mechanize I think I'm stuck with it's logging. I can't run WireShark as the traffic is HTTPS and encrypted. Any ideas?

Q3 - If the exact issues isn't obvious any suggestions/ideas re what to try?

FURTHER BACKGROUND:
==================
* Successful Trace - manually via browser - using HTTP Live Headers plugin for Firefox (I've shorted session key for succinctness) POST /IBAU/BANKAWAYTRAN;jsessionid=xxx Action.RetUser.SignonOK.x=39&Action.RetUser.SignonOK.y=12&Action.RetUser.SignonOK=Proceed+to+Internet+Banking

* Unsuccessful Trace - i.e. What I see in the Mechanize log file Net::HTTP::Post: /IBAU/BANKAWAYTRAN;jsessionid=xxx

* Guts of the Form I'm simulating the response for:
<FORM ACTION="BANKAWAYTRAN;jsessionid=xxx" METHOD="POST" NAME="securityForm"> <INPUT TYPE="Image" SRC="web/L001/images/00010/b_proceedIB_99CC00.gif" VALUE="Proceed to Internet Banking" alt="Proceed to Internet Banking" NAME="Action.RetUser.SignonOK"> </input> </FORM>
* My Mechanize Code

require 'rubygems' require 'mechanize' require 'mechanize_extns/form_extn.rb' require 'logger' agent = WWW::Mechanize.new{|a| a.log = Logger.new(STDERR) } # Step 1 - Navigate to Login Page page = agent.get('https://www.anz.com/INETBANK/login.asp') if !page.forms[1] || (page.forms[1].name != "loginForm") puts "ERROR: At login form validation failed" exit end # Step 2 - Login login_form = page.forms[1] login_form.CorporateSignonCorpId = "xxxx" login_form.CorporateSignonPassword = "yyyy" page = agent.submit(login_form) if !page.forms[0] || (page.forms[0].name != "securityForm") puts "ERROR: At clicking on continue button after login" exit end # Step 3 - Click Continue to Main Page (** WHERE ISSUE OCCURS **) continue_form = page.forms[0] # work around 1 - the bank's input tage of type image does not seem to get handled, so add a button + in continue_form.add_field!("Action.RetUser.SignonOK","Proceed to Internet Banking") # Don't see +effect of this in log trace? continue_form.add_field!("Action.RetUser.SignonOK.x","23") # Don't see +effect of this in log trace? continue_form.add_field!("Action.RetUser.SignonOK.y","7") # Don't see +effect of this in log trace? # work around 2 - the relative URI in the bank form's action does not seem to get handled, so add t +he front part on continue_form.action = "https://banking2.anz.com/IBAU/" + continue_form.action page = agent.submit(continue_form) if !page.forms[1] || (page.forms[1].name != "formAll1") puts "ERROR: At clicking on continue button after login ==================================" # +== FAILED HERE WITH SESSION ENDED IN REPONSE puts page.content puts "ERROR: At clicking on continue button after login ==================================" exit end # Step Final - Logout puts "Completed!"

Any other info that would help you out helping me out?
Direct Responses: Write a response