I need to run 1000s of mysql select queries and save the results in different files (1 file per query) everyday. Currently every mysql select query is running as a background command through shell script as follows
echo "select (CART_ID), '', (USER_ID) from shopping_cart where (
DATE_MODIFIED >= '2018-01-08 00:00:00'
and DATE_MODIFIED < '2018-01-10 00:00:00'
)" | mysql -u <usr_name> -p<password> -h <host> -P 3306 \
-D <schema< -C -N --default-character-set=utf8 > \
Currently, there are around 5 shell script files; each containing ~250 similar queries writing into 250 different files. So, there will be ~1000 files everyday.
Requirement is not to join the queries to produce unified results.
Is there a better way to do in Java? Googling says its expensive to open mysql connection for each query (thread) and java program to run mysql command shell script is not working as expected.