Hi there,
I have a middleware platform which has jobs scheduled to scrap data from a website by calling the same Java service which uses HtmlUnit (by passing different search parameters to the web page). Therefore, it might happen that multiple separate threads are used with HtmlUnit.
The problem is that method getPage returns the same old page no matter what I set in the code. I tried to use below configs:
client.getCache().clear();
client.getCache().setMaxSize(0);
client.getOptions().setCssEnabled(true);
client.getOptions().setJavaScriptEnabled(true);
No matter what I do, below code always returns the same page:
page = client.getPage(webPageURL);
I also tried the method refresh() but it just won't work.
Can you please tell me what I am missing?
Best regards,
n23
Hi there,
I have a middleware platform which has jobs scheduled to scrap data from a website by calling the same Java service which uses HtmlUnit (by passing different search parameters to the web page). Therefore, it might happen that multiple separate threads are used with HtmlUnit.
The problem is that method getPage returns the same old page no matter what I set in the code. I tried to use below configs:
client.getCache().clear();
client.getCache().setMaxSize(0);
client.getOptions().setCssEnabled(true);
client.getOptions().setJavaScriptEnabled(true);
No matter what I do, below code always returns the same page:
page = client.getPage(webPageURL);
I also tried the method refresh() but it just won't work.
Can you please tell me what I am missing?
Best regards,
n23