Search This Blog

Breaking

Friday, 20 January 2023

January 20, 2023

How I reduced 9 hour execution time to 1 hour with Java threads and ChatGPT Help?

 Recently, I was writing a utility which is comparing data from 3 different data sources. Here the first source was UI, second was Web Services and third was Excel where Data was dumped through some sequel queries. 













As the data (UI and Services) which needs to validate against was Excel data, So my initial implementation was:

1. To read data from excel and store it inside a HashMap

2. Hit the UI for the fund data read from excel with JSOUP library and store the data

3. Hit the Services with JSOUP and store the data

After getting the data, I was performing 2 comparisons:

-> UI data to services

-> UI Data to Excel data

When I ran it for almost 2200+ funds, it took me almost 8 hour to complete the comparison.

I checked the logs and find out that:

1-> Getting data from UI was taking ~7-9 seconds from JSOUP

2-> Rest all was completing within 1 second


As I have all the funds name already available with me when I read it first from excel, and after that, 1 by 1 I was making UI call for those funds, so I thought to implement Thread concept here.

So I built a new approach for which ChatGPT helped me.

I asked a question in chatGPT something similar to below:

-> Provide me a sample java code to create no of threads as per the array list size and later execute them all altogether.


ArrayList<String> values = new ArrayList<>();
values.add("a");
values.add("b");
values.add("c");
values.add("d");
ArrayList<Runnable> tasks = new ArrayList<>();

for(String value: values) {
tasks.add(()-> {
System.out.println( " Executing Task for " + value); // Add you method here
});
}
ExecutorService executor = Executors.newFixedThreadPool(tasks.size());
for (Runnable task : tasks) {
executor.execute(task);
}
executor.shutdown();
}


1-> After reading the fund names from excel, I will create task for all these funds which will hold method to read data from UI.

2-> After creating task, I will execute these task with Thread Pool.


So My implementation code changed to: (Dummy Code)


This saved the overall execution to 1/3rd drastically. I used all the available resources (machines in Jenkins) and created a parallel pipeline also and further reduced this 1/3rd to further 1/3rd which means I brought 9 hour execution to 1 hour.

This was the best way I learned a concept in my life and chatGPT helped me a lot in it.

Tuesday, 10 January 2023

January 10, 2023

How to in Git?

 1. How to remove a file from a remote feature branch without deleting it from local branch?


1. Checkout to the local branch and run below command:

>git rm --cached sampletextfile.txt




2. run git status. It will show 


Changes to be committed:

  (use "git restore --staged <file>..." to unstage)

        deleted:    sampletextfile.txt

3. Now commit the changes

git commit -m "pushing changes"


4. After commit , perform git push.

gitDemo>git push origin HEAD

2. How to Create and checkout the new branch with one command in git?

>$ git checkout -b test origin/test

where origin/test is the remote branch name and test is branch name in local.


Note: Always take a fresh pull from the main branch first, so that this new branch is created with latest changes.


3. How to delete a remote branch? (Assuming you have the permission)

-> Check all the branches

git remote show origin

-> Delete the specific branch

git push origin --delete <branch-name>