In UI Automation, We have many times observed script failures when a page or an element or an element value does not load within time and your test fails with an exception “element not found “. We usually call these failures as selenium script failures. Many times, when we re-run it, it passes without the same issue.

The simple solution, we put in this situation since ages are to play with your time which you have given to load the page/element. We usually increase the page/element load time and hurray, we solved the problem.

Everyone is happy because the report is green again.

But did we solve the actual problem here or unknowingly we have hidden some issues?

Now let me share my experience.

A few years back, I was asked to resolve a prod smoke automation failure by the PO for a script that was created by someone else. The script was failing intermittently and making the report “Red”. :)

I was told to increase the timing in scripts so that this intermittent issue will get resolved and reports will start coming as “green”.

I checked the script and observed that there is already a 50 sec time was present. I increased it to 1 minute.

I re-ran it and it passed. I again re-ran and it again passed. I did this 5 time and it passed in all those 5 times.

So the solution was to increase 10 seconds there. hmm.

But I was skeptical about it as It was intermittent. So I re-ran it 5 more times and now, one more time it failed.

I explained this to the product owner and he told me to increase the time one more time. :)

I asked him, Can you suggest to me the time, which it should take to reflect on the page? As he was not aware of the current timeout present in the automation script, he responded that it should be a maximum of 30 sec.

I showed him the logs and told him that it is taking one minute in some particular circumstances. I told him, if 30 sec is the max time, then this is a performance issue.

We started digging the logs(This time application), included the tech lead also, and checked why it took so much time. It came out that this particular element value comes from the back end. We checked the API calls and identified some areas of improvement in the services.

Later, we all agreed, that if it will take more than 30 sec, then this issue will not be fixed from Automation, it will be logged as a performance issue.

So next time, if you face this situation, and someone says you increase the timeout, then ask them, how much and why this much? Should we log a performance defect if it goes beyond that?

#UIAutomation #Selenium #Webdriver