How I reduced 9 hour execution time to 1 hour with Java threads and ChatGPT Help?
Recently, I was writing a utility which is comparing data from 3 different data sources. Here the first source was UI, second was Web Services and third was Excel where Data was dumped through some sequel queries.
As the data (UI and Services) which needs to validate against was Excel data, So my initial implementation was:
1. To read data from excel and store it inside a HashMap
2. Hit the UI for the fund data read from excel with JSOUP library and store the data
3. Hit the Services with JSOUP and store the data
After getting the data, I was performing 2 comparisons:
-> UI data to services
-> UI Data to Excel data
When I ran it for almost 2200+ funds, it took me almost 8 hour to complete the comparison.
I checked the logs and find out that:
1-> Getting data from UI was taking ~7-9 seconds from JSOUP
2-> Rest all was completing within 1 second
As I have all the funds name already available with me when I read it first from excel, and after that, 1 by 1 I was making UI call for those funds, so I thought to implement Thread concept here.
So I built a new approach for which ChatGPT helped me.
I asked a question in chatGPT something similar to below:
-> Provide me a sample java code to create no of threads as per the array list size and later execute them all altogether.
ArrayList<String> values = new ArrayList<>();
values.add("a");
values.add("b");
values.add("c");
values.add("d");
ArrayList<Runnable> tasks = new ArrayList<>();
for(String value: values) {
tasks.add(()-> {
System.out.println( " Executing Task for " + value); // Add you method here
});
}
ExecutorService executor = Executors.newFixedThreadPool(tasks.size());
for (Runnable task : tasks) {
executor.execute(task);
}
executor.shutdown();
}
1-> After reading the fund names from excel, I will create task for all these funds which will hold method to read data from UI.
2-> After creating task, I will execute these task with Thread Pool.
So My implementation code changed to: (Dummy Code)