Recently someone posted a challenge in linked to fill value inside a field. It looks a simple problem as from UI page, it was just a normal input field. I tried to check the dom for the element, and got to know that this is inside closed shadow dom.

So if you also find an element inside shadow dom which is closed, it means it can not be interactable. So the first approach to solve such issue is to connect with your developer teams. Yes, they can provide you an alternate way to access that element (at least in lower environment.)

But if this is not possible, then use selenium keypress method by using a common use case of an end user.

The same way I solved that challenge also. :) 

So, Just think, that this field is not accessible from mouse. In this situation, usually we interact on the page items with keyboard "TAB" key. Same approach, we can take here as well.

We just need to find an element which is accessible from selenium usual locator finding techniques, bring focus to that element, sometimes just by clicking on it or if it is an input field, just by passing a empty string, then press tab key until you reach to that element.

Bingo, you are now inside that field, now if you need to perform:

1. Click on that element, press "Enter" key

2. If it is an input field, you can simply use "send_keys" method

To demonstrate it, I created a sample page having shadow root as closed in it.

In the above page, the "Age field " is not inside any shadow dom element. And the first name and last name fields are inside the shadow dom[closed].

So to enter value inside the firstname field, I entered value or bring my focus to "Age field " first. And then I used "Tab Key" which brought me to the "First name" field. 

Once, My focus is available in the first field, I used action class to send my input to that field.

driver.get('file:///C:/Users/username/Documents/shadowDom.html')

driver.maximize_window()

ageElement = driver.find_element(By.ID, 'age')

ageElement.send_keys(30)

ageElement.send_keys(Keys.TAB)

actions = ActionChains(driver)

actions.send_keys("My name")

actions.perform()

sleep(10)

driver.quit()

Check the other post, how to interact with an element which is inside shadow dom [Open] tag.

Recently, I saw in stack overflow that many people were posting question that they are not able to get element which is inside a shadow DOM. So I tried to solve this problem. Below is my learning:

1. If the element is inside the shadow root, first check that is that shadow root is open or close. If it is Open, than you can directly access it by using java script executor.

2. If the element is inside the shadow root which is closed, you can not use it. If you need to interact with such element, you can use keyboard key press operations.

So the first application which I got was an application where the shadow root was open. So it became easy for me. My take here is, first check your locator in developer console. I will explain how we can do it.

So lets first open a page  where this issue occurs . And the easy to accessible page is your chrome download page. If you will inspect "Search Downloads"  element on that page, you will find that it is inside shadow root.

So if we try to find element by using xpath, the xpath value is : '//*[@id="searchInput"]

But if you will try to search this XPATH, you will find it is not highlighting the element. And if you use it in your code, then you will see below error:

So the reason for it's not working is not that your xpath is not correct, it's just that this element is present inside shadow root element.

And if any element is present inside shadow root [Open], you can not access it with normal locator strategies. 

So to access this element, you will have to first find css locator for the topmost element from where the shadow root starts. In this page, it is from the beginning. So just find the locator of the element which is before the shadow root.

Here it is : body > downloads-manager

Now, you will move to find the second shadow root and you need to find the locator of element before that element.

Here it is: #toolbar

So this is how you can enter text inside a shadow root element.

If we have a pipeline job and if we want to trigger build based on some conditions, we can use if -else statement in the pipeline. I faced this situation for my master parallel job which was used for calling other jobs. Now, I need to call those jobs based on the choices or environment, so I try to put an if- else condition there.

Below is one sample pipeline, which executes code based on your choice parameter.

pipeline {

    agent any

    tools {

        maven 'Maven3'

        jdk 'java8'

    }

     parameters {

        choice(choices: ['QA', 'stage', 'dev'], description: 'Which Env?', name: 'PickYourEnv')

    }

    stages {

        stage('Build') {

            steps {

                 script {

                     echo "This is environment name : " + String.valueOf(params.PickYourEnv)

                       currentBuild.description = "${params.PickYourEnv}"; 

                    if (String.valueOf(params.PickYourEnv) == 'QA') {

                        echo 'I only execute on the master branch'

                        git 'https://github.com/qamatters/KarateDemo.git'

                        bat "mvn clean test -DargLine='-Dkarate.env=e2e' -Dtest=CucumberReport -DfailIfNoTests=false"

                    } 

                    else {

                        echo 'I execute elsewhere'

                    }

            }

    }

            post {

                // If Maven was able to run the tests, even if some of the test

                // failed, record the test results and archive the jar file.

                success {

                    cucumber buildStatus: 'null', customCssFiles: '', customJsFiles: '', failedFeaturesNumber: -1, failedScenariosNumber: -1, failedStepsNumber: -1, fileIncludePattern: '**/target/karate-reports/*json', pendingStepsNumber: -1, skippedStepsNumber: -1, sortingMethod: 'ALPHABETICAL', undefinedStepsNumber: -1

                }

            }

        }

    }

}

Recently, I need to re-install Jenkin in my local machine because of some issues. I didn't save the local data copy, so I have to start freshly again when I re-installed Jenkin. One more issue I observed, that from my machine JDK is also removed.

I installed Jenkin (it needs JRE/JDK), navigated to Global tool configurations from Manage Jenkin link, and added my (oracle) cred to download JDK (Oracle cred is needed to install Jenkin).

Then, I added maven.

I saved these changes. Then I create a new pipeline job.

When I ran that pipeline job, I started getting this issue, "Maven: WARNING Unable to autodetect 'javac' path, using 'javac' from the environment". As Java is installed from Jenkin itself, I was expecting it to figure out the path as well.

I checked my pipeline script and observed that, I missed the tools section in it.

Once I added it, it identified the installed java and removed the above mentioned error.

Recently I started an activity to answer stack overflow question tagged with Selenium. To my surprise, most of the question were coming for scrapping website contents with the help of selenium and python.

And most of the question were having the same problem, that was that they were struggling with pagination. I answered few questions and below is my one code snippet from ebay site scrapping for apple products.

Tip: Always check how your URL is constructing when you are talking about pagination. Most of the time, your URL will be appending page no which will help you to loop.

from time import sleep

from selenium.webdriver.common.by import By

from selenium import webdriver

PATH = r" Your chromdrive exe path"

driver = webdriver.Chrome(PATH)

url_first_part = "https://www.ebay.com/b/Apple/bn_21819543"

for i in range(5):

    i = i + 1

    url = url_first_part + "?_pgn=" + str(i)

    driver.get(url)

    sleep(3)

    driver.maximize_window()

    products = driver.find_element(By.XPATH, '//*[@id="s0-27-9-0-1[0]-0-1"]/ul')

    print(products.text)

driver.quit()

Output: Now it will navigate to each page (till 5) and will scrape the content.

 Recent ally, I encountered one issue where one of the multilingual texts

was coming differently in my local environment(IDE). It was a pretty annoying issue because the string was coming with special characters when I was running my program, while when I tried to replicate it in a different project, the same was working fine.

I posted the question in StackOverflow too but could not find a better solution for it.

To fix this issue, I used apache common lang3 Stringutil escape until method. This method converted the special characters to HTML. Now, I thought to put a regex to remove the HTML. But with my bad, it didn't work.

So, after spending a lot of time, I figured that this is happening because of encoding. So I checked my file encoding.

In Intelij, we can check encoding by: ctril+shift+d -> Editor->File Encoding.

There I found that the file encoding was set as "Windows 1252". 

I changed it to UTF-8. Later, I change all individual file encoding also to UTF-8.

This fixed the issue in my local machine.

The same issue when I got in Jenkin machine, I fixed the encoding from the pom.xml file.

Here is my question from StackOverflow.

 If the click() in your selenium UI script is failing in Jenkin machine, but in local, it is working fine, then you can use below tricks:

1. Replace selenium click() to JavaScript Executor click()

JavascriptExecutor javascriptExecutor =  (JavascriptExecutor) driver;

javascriptExecutor.executeScript("arguments[0].click();", element);

2. Scroll till that element first, and then perform click.

JavascriptExecutor javascriptExecutor =  (JavascriptExecutor) driver;

javascriptExecutor.executeScript("arguments[0].scrollIntoView();", element);

3. Use double click

element.doubleClick()

4. Use isDisplayed(), isClickable(), waitForPageToLoad() method. It will buy some time before clicking on that element.

5. Use Hardcode for debugging purpose. Check that, is it failing because element is not visible.

6. Try to do getText() for that element. If the locator is working, it should return text for that element also if it's present.

7. Try to find out window size of that jenkin machine where your code is running. There are chances, that the browser size is different in your local to Jenkin machine.

You can use below code for that:

Dimension dimension = driver.manage().window().getSize();

LOGGER.info("dimension is " + dimension);

Later set the size.

driver.manage().window().setSize(new Dimension(1280, 768));

8. Use fullScreen, maxsize for browser.

driver.Manage().Window.Maximize();

 Today I encountered a weird issue where in my local. I am able to retrieve element text with the help of selenium getText() method while in jenkin machine, it was giving empty value for the element text.

To fix this issue, rather then getting the text via getText() method, I used getAttribute method.

And I gave the property as "innerText".

So I replaced my getText() to getAttribute("innerText") and hurray, It returned the text now.

if this also doesn't work for you, try with below options:

1. getAttribute("innerHTML")

2. getAttribute("textContent")

This saved my lot of time because I was using scrollView method else to navigate that element first and then retrieve the text value.

 When you start enhancing your karate framework, there comes a time when you want to read your auth token information from your config file, so that it can be set once, and reused in entire test suite run.

If you are in this stage or even if you are seeing how you can read your auth token from config file, then this post will help you.

So initially, I was having a token generation feature file which I was calling in my feature file with the help of callSingle method in background.

* def result = karate.callSingle('classpath:TokenGeneration.feature')

*def token = result.token

* def tokenId = result.tokenId

Once I get the otken and tokenId, I can use it in my feature file.

And header token = token

And header tokenId = tokenId

Later, I thought to put this token feature inside karate config file, so that these values will be set once, and we can reuse it again.

So, for this, we need to go to karate config file and at the bottom, before return config, call this token feature.

var result =  karate.callSingle('classpath:TokenGeneration.feature', config);

authInfo = {authToken:result.token, authTokenId: result.tokenId};

return config;

Now in your feature file, you can use it like in this manner:

*def token =authInfo.authToken

* def tokenId = authInfo.authTokenId

Though I was very impressed last time when I used webdriverIO visual automation capability, but the flakiness which comes with it, always made me skeptical about relying on it much. And now, when I am again back to java based automation framework, I feel it to give it a try one more time. 

So after searching about the library which can give support the visual capability for a java based project, I came across the Ashot library. I added its dependency in my maven project and started working on a POC. I tried to do a comparison for a static page, specific web element and then a page having dynamic content.

In my local machine, I didn't face any issue, and it worked great. But the one thing, which disappointed me is their support. Looks like, team is not actively maintaining the code. And it might be, that if you will stuck at some place, you wouldn't find support much. 

So if you have a java maven project, add Ashot maven dependency.

<dependency>

    <groupId>ru.yandex.qatools.ashot</groupId>

    <artifactId>ashot</artifactId>

    <version>1.5.4</version>

</dependency>

 Recently I feel the need to have a static code analysis of my one of the java automation project. I was aware about the sonar qube and sonar lint but I never implemented or used these tool in any of my automation project. Though in one of my Typescript bases project we have used TSLint which was very impressive. 

So I started googling about it. And picked one of my sample project to use these tools. Let me first differentiate between these 2. 

Sonar lint lives only in the IDE (IntelliJ, Eclipse and Visual Studio). Its purpose is to give instantaneous feedback as you type your code. For this, it concentrates on what code you are adding or updating.

SonarQube is a central server that processes full analyses (triggered by the various SonarQube Scanners). Its purpose is to give a 360° vision of the quality of your code base. For this, it analyzes all the source lines of your project on a regular basis.

So I preferred to use sonar lint in one of my project. As usual, I added this in my favorite IDE intelij. Once it get installed, you can see an option sonar Lint in the bottom as an option,

It keeps showing you code smells with explanation and a possible solution. 

For example:

This helps you to reduce lots of small issues which we simply missed because of unawareness and our ignorance. We can run this for the whole project also, though it takes some time to get complete if its a large project.

Sonar lint is available for eclipse and visual studio as well.

 Recently, I encountered a weird issue where in my local machine, all my UI Automation scripts were working perfectly fine but when same ran on Jenkins machine, they started failing by giving exception: Element is not clickable at this point. I first thought, it might be happening because of different resolution of Jenkin machine (which still a possible reason) but unfortunately, I didn't have access to that Jenkin machine where it was failing.

To fix this, I tried to change the selenium click method to java script executor click method. And, it worked.

WebElement elem = driver.findElement(By.xpath("your xpath"));

JavascriptExecutor executor = (JavascriptExecutor) driver;

executor.executeScript("arguments[0].click();", elem);

So the second thing which I did was changing all click method to java script executor click. But then I identified, that few of the clicks are not working.

Those were the elements which were present as Button in the webpage. So I didn't change that click.

And It resolved.

 This post is in continuation of my last worlfront post. So after creating request, the next operation is to validate that with whatever field values, we have created the request, same is reflecting there or not. The same validtaion, we need to perform after converting this request to project as well. 

So to perform both these validations, we need to fetch the field values from API response. Below are the end points which can be sued for fetching the parameter values from a request/task:

Method: Get

Path: /attask/api/v9.0/optask

Params: action: generateApiKey

Param: apiKey: apikeyvalue

Param: ID: requestid

Param: fields:parameterValues

Similarly, if we need to fetch field values from project, then we need to replace ID with projectID, though end point will change.

Method: Get

Path: /attask/api/v9.0/project/search

Params: action: generateApiKey

Param: apiKey: apikeyvalue

Param: ID: projectID

Param: fields:parameterValues

Similarly, if you want to fetch the custom form names for a project, you can get it with below details:

Method: Get

Path: /attask/api/v9.0/PROJ/search

Params: action: generateApiKey

Param: apiKey: apikeyvalue

Param: ID: projectID

Param: fields:objectCategories:category:name

This will give all the custom form name for the project.

 If you are using zoom APIs and at some time, you need to create meetings via API's, then the below endpoints can help:

1. First create a token. Here, I am using OAuth2.0. To create Oauth2.0, we can use client id and secret id.

It might be that sometime in a project, someone will say, you to take a clone from a specific branch (not from master) and will request you to push your changes or raise PR for that branch. In such situation, below commands can help you:

1. First check all the branches in remote 

git branch -r

It will show you all the branches available in remote. It should have the requested branch also.

2. For example, here I am creating a new branch development.

What it does is: It creates a local branch with the name specified by you which will track the remote branch specified by you. 

 In my last project, I worked on a SAAS tool "Workfront". Workfront provides a software-as-a-service platform for enterprise workflow management, project and portfolio management, resource management and individual task management. Workfront's platform automates and repeatable processes and allows users to manage financial information and create reports.

When I got an opportunity to automate our customized Workfront application, I learned a lot about Workfront API's. This article is about my learning of those API's (Which I could not find easily in internet).

If you are also in such a situation, then the first step to start understanding Workfront API's are Workfront API documentation page. Here, you will get a lot of information about Workfront objects and their API details.

So When I started working on it, I started with one of my Workflow. Let me describe my workflow a bit:

1: Create a request (Task)

2. Validate all field presence in request

3: Change it's assignment and add a user 

4: Change its status

5. Convert it to a project 

6. Validate project having the same data which we submitted at the time of request creation

7. Validate that the converted project is having the same task list which is already defined for this task template

8. Add document to project

9. Mark all the task as complete in Workfront

So to perform this workflow, I picked Workfront API's. To work on Workfront API, first I created an appKey. Take help from your team who manages Workfront accesses.

Between, below is the point for creating/getting the appkey:

1. Create an appkey for new user:

Method: Put

Path: /attask/api/v9.0/user

Params: action: generateApiKey

Param: username: username

Param: password: password

Param: method:put

2. Get an alredy created appkey for a user:

Method: Put

Path: /attask/api/v9.0/user

Params: action: getApiKey

Param: username: username

Param: password: password

Param: method:put

Once you will have the appkey, for creating a request, you can use /OPTASK endpoint.

1. Create Request/Task End Point: 

Method: Post

Path: /attask/api/v9.0/

Params: appKey

Body: Request Body

2. Validate all field presence in request:

Method: Get

Path: /attask/api/v9.0/

Params: appKey : appkey value

Params: ID : requestID

Params: fields : *

3.  Change it's assignment and add a user 

Method: Put

Path: /attask/api/v9.0/

Params: appKey : appkey value

Params: ID : requestID

Body: Request body containing userid

4. Change its status

 Method: Put

Path: /attask/api/v9.0/

Params: appKey : appkey value

Params: ID : requestID

Params: status: RPC (Ready for project conversion)

5. Validate Project Creation:

 Method: Get

Path: /attask/api/v9.0/project/search

Params: appKey : appkey value

Params: name : requestname

fields: *

6. Get all task for a project:

 Method: Get

Path: /attask/api/v9.0/task/search

Params: appKey : appkey value

Params: projectID: projectID

7. Get all task from a template:

 Method: Get

Path: /attask/api/v9.0/templateTask/search

Params: appKey : appkey value

Params: templateID: templateIDValue

8. Create a folder: (Later add document to it):

 Method: Post

Path: /attask/api/v9.0/docfdr

Params: appKey : appkey value

Params: name: docFolderName

Params: userID: userIDValue

9. Add document to folder

 Method: Post

Path: /attask/api/v9.0/document

Params: appKey : appkey value

Body: Request body containing file information

10. Mark all the task as complete in Workfront

 Method: Get

Path: /attask/api/v9.0/task/

Params: appKey : appkey value

Params: projectID: projectIDValue

Param: name: projectName

Param: taskNumber: taskNumberValue

Param: percentComplete:percentCompleteValue

While Doing this activity, I explored many other Workfront end points also. This I will update in some time.

If you are trying to read paraquet file, then it can be done by adding these 2 dependencies in pom file.

<dependencies>

 <dependency>

 <groupId>org.apache.parquet</groupId>

 <artifactId>parquet-hadoop</artifactId>

 <version>1.9.0</version>

 </dependency>

 <dependency>

 <groupId>org.apache.hadoop</groupId>

 <artifactId>hadoop-common</artifactId>

 <version>2.7.0</version>

 </dependency>

</dependencies>

If you are planning to start doing coding practice in typescript, then the below steps can help you for doing the initial setup.

1. First, install visual studio (windows installer)

2. Install node (windows installer)

3. Check node --version (visual studio new terminal)

4. Once you have a node, you can use npm. So with the help of it, install typescript globally. 

npm install -g typescript (visual studio new terminal)

5. Check the typescript version to make sure that typescript is installed or not. 

tsc --version

Version 4.3.4

or (You might face this issue)

tsc : File C:\Users\deepak.mathpal\AppData\Roaming\npm\tsc.ps1 cannot be loaded because running scripts is disabled on this system. For more information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.

At line:1 char:1

+ tsc --version

tsc.cmd --version

6. To fix the above issue, open powershell with admin (visual studio powersheell does not open with admin access)

provide the access:

Open Commandform : windows + R

Type : Powershell

Then type : set-executionpolicy remotesigned

And select opetion : A

7. now, you can run tsc command. 

tsc --version

Version 4.3.4

8. Create a folder and open it in vscode

C:\Users\deepak.mathpal\Documents>mkdir typescript

C:\Users\deepak.mathpal\Documents>cd typescript

C:\Users\deepak.mathpal\Documents\typescript>code .

C:\Users\deepak.mathpal\Documents\typescript>

9. From the File Explorer, create a new file called helloworld.ts.

let message: string = 'Hello World';

console.log(message);

10. To compile your TypeScript code, you can open the Integrated Terminal (Ctrl+`) and type

 tsc helloworld.ts. 

This will compile and create a new helloworld.js JavaScript file.

11. If you have Node.js installed, you can run node helloworld.js.

12. tsconfig.json

So far, we have been relying on the TypeScript compiler's default behavior to compile our TypeScript source code. We can modify the TypeScript compiler options by adding a tsconfig.json file that defines the TypeScript project settings such as the compiler options and the files that should be included.

Add a simple tsconfig.json which set the options to compile to ES6 and use CommonJS modules.13.

13. {

    "compilerOptions": {

      "target": "es6",

      "module": "commonjs",

      "outDir": "out"

    }

  }

  14. add above code in tsconfig file, delete the js file eaerlier created, open a new terminal and run again. Now js file will be inside out directory

  15. Run code now from out directory.

References:

1. https://stackoverflow.com/questions/58796490/tsc-ps1-cannot-be-loaded-because-running-scripts-is-disabled-on-this-system

2. https://code.visualstudio.com/docs/typescript/typescript-tutorial

If we want to run selenium execution in an open browser, (in mac machine) we need to first find out our chrome location in mac. Mostly, it remains under the Application folder if you have not done any specific changes while installing the browser. 

Once you get the location, create a new folder at any location and copy its path. 

I created an in-home location. 

Once this is done, run this command in the terminal:

% /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 -user-data-dir=chromedata

It will open a new chrome browser, where we can debug our fix. Now in your automation script, put a option for chromedriver debugging.

 options.setExperimentalOption("debuggerAddress", "localhost:9222");

Now Provide this option to your chromedriver.

In My case, it was something like this:

capabilities.setCapability(ChromeOptions.CAPABILITY, options);

 System.setProperty("webdriver.chrome.driver", chromeDriverPath);

  driver = new ChromeDriver(capabilities);

 Now when you will run your script it will run in this already open browser.

Note: Port number, directory location is optional.

For windows, check this link: Selenium Script execution on already open browser Windows OS

 This is a part of my learning series where recent ally I read an article about how to execute selenium script in an already open browser. This is needed when some of our test cases get fails and we put a fix. When we want to test that fix, then our script starts from scratch and this process becomes tedious when our fix doesn't work and we again and again put our fixes. A lot of time spends in unnecessary execution of same steps which are already working.

So, to avoid this situation, and to run our execution exactly from where it's failing, we can use chrome browser native debugging option. 

What we need to do to achieve this is, we need to launch our chrome in debugging mode. Below are the steps which we can follow to achieve this:

1. Find the chorome.exe in your machine. In my case, it is in, C:\Program Files\Google\Chrome\Application.

2. Create a directory in your machine. I created C:\Users\deepak.mathpal\Documents\chromedata

3. Now run this command.

C:\Program Files\Google\Chrome\Application>chrome.exe --remote-debugging-port=9222 --user-data-dir=C:\Users\deepak.mathpal\Documents\chromedata

4. It will open a new chrome browser, where we can debug our fix.

5. Now in your automation script, put a option in your chromedriver for debugging.

 options.setExperimentalOption("debuggerAddress", "localhost:9222");

6. Provide this option to your chromedriver.

In My case, it was something like this:

capabilities.setCapability(ChromeOptions.CAPABILITY, options);

 System.setProperty("webdriver.chrome.driver", chromeDriverPath);

  driver = new ChromeDriver(capabilities);

7. Now when you will run your script it will run in this alredy open browser.

Note: Port number, directory location is optional.

References: 

1. https://chromedevtools.github.io/devtools-protocol/

2. https://www.youtube.com/watch?v=Rrkj4tdXngY

Recently, I came across a UI Automation scenario where I need to validate the clipboard behavior of a field. 

The scenario was: I need to click on a copy SQL button which copies a large SQL which I need to validate. Earlier, I have used the ctrl+c and ctrl+v approach, but here, there was no text field where I can paste that content.

So, at run time, I need to validate the content has been copied or not.  

To automate this, I used the java clipboard class. 

Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();

Once the clipboard object is created, you can click on the element.

driver.findElement(By.cssSelector("div:nth-child(2) > .input-group .octicon-clippy")).click();

Thread.sleep(2000);

Now, you can extract this data.

Object dataFromCopyURLFeature = clipboard.getData(DataFlavor.stringFlavor);

System.out.println("Copied data is " + dataFromCopyURLFeature);

You can assert this data, with the actual data for validation purpose.

Assert.assertTrue(dataFromCopyURLFeature.equals("https://github.com/qamatters/KarateDemo.git"), "Validate the url");

Let me know if code is needed for this.