Search This Blog

Breaking

Tuesday, 28 February 2023

[JSOUP] [Fix]-> Not fetching whole page data from page

Recently, I encountered an issue with our JSOUP bases automation project where, we were not getting whole page data in the document.

Document doc = Jsoup.connect(performanceTabURL).get(); 











When I printed this doc, I didn't see the whole page data. Now the intresting part of this issue was, that the problem was coming in 1 machine/network (client) while in another network, this was fetching the all data.

Though, I could not still figure out why this is happening in 1 machine/network, but to fix this problem, I increased the size of the body in jsoup request, which by default is set to hold for only 1 MB size of data.

So I tweaked the above code and added maxBodySize which takes size as an integer value.

Document doc = Jsoup.connect(performanceTabURL).maxBodySize(0).get();

Size as 0 means, that it will not have restriction on size.

With this change, I waas able to fetch the whole data without worrying the size of the body.

No comments:

Post a Comment