Submitting a Form using Selenium in Python

Question

I need to scrape some data behind those hyperlinks from this Site. However, those hyperlinks are javascript function calls, which later submits a form using post method. After some search, selenium seems to be a candidate. So my question is that how should I properly set a value to an input tag and submit the form which does not a submit a button.

from selenium import webdriver

url = "http://www.echemportal.org/echemportal/propertysearch/treeselect_input.action?queryID=PROQ3h3n"
driver = webdriver.Firefox()
driver.get(url)
treePath_tag = driver.find_element_by_name("treePath")

Before submitting the form, I need to assign value to tag <input>. However, I got an error

Message: Element is not currently visible and so may not be interacted with

treePath_tag.send_keys('/TR.SE00.00/QU.SE.DATA_ENV/QU.SE.ENV_ENVIRONMENT_DATA/QU.SE.EN_MONITORING')

IF above is correct, I would like to submit form this way. Is it correct?

selenium.find_element_by_name("add_form").submit()

Below are sources from the web page.

JavaScript function

<script type="text/javascript">
    function AddBlock(path){
        document.add_form.treePath.value=path;
        document.add_form.submit();
    }
</script>

form "add_form"

<form id="addblock_input" name="add_form" action="/echemportal/propertysearch/addblock_input.action" method="post" style="display:none;">
<table class="wwFormTable" style="display:none;"><tr style="display:none;">
  <td colspan="2">
<input type="hidden" name="queryID" value="PROQ3h1w" id="addblock_input_queryID"/>  </td>
</tr>
<tr style="display:none;">
  <td colspan="2">
<input type="hidden" name="treePath" value="" id="addblock_input_treePath"/>  </td>
</tr>
</table></form>

div with javascript call

<div id="querytree">
    <h1>Property Search</h1>
    <h2>Select Query Block Type</h2>
    <p>Select a section for which to define query criteria.</p>
    <div class="queryblocktools"><a href="javascript:document.load_form.submit();"><img style="vertical-align:top;" alt="Load" src="/echemportal/etc/img/load.gif"/>&nbsp;Load Query</a></div>
    <ul class="listexpander">   
    <li>Physical and chemical properties<ul>
        <li><a href="javascript:AddBlock('/TR.SE00.00/QU.SE.DATA_PHYS/QU.SE.PC_MELTING');">Melting point/freezing point</a></li>
        <li><a href="javascript:AddBlock('/TR.SE00.00/QU.SE.DATA_PHYS/QU.SE.PC_BOILING');">Boiling point</a></li>
    </ul>
</div>

Solution

You are trying to set value on hidden input is which not visible on the page, that's why error has occurred. If you want to set value on hidden field try using execute_script as below :-

treePath_tag = driver.find_element_by_name("treePath")
driver.execute_script('arguments[0].value = arguments[1]', treePath_tag, '/TR.SE00.00/QU.SE.DATA_ENV/QU.SE.ENV_ENVIRONMENT_DATA/QU.SE.EN_MONITORING')

After setting value on hidden field you can use following to submit the form :-

selenium.find_element_by_name("add_form").submit()

Hope it helps..:)