u/Ok_Egg_6647

Hi all
I need a way by which i can store logs presistely
My log which currently only displayed over terminal are like this

16:47:40 │ INFO │ app.infrastructure.postgres.candle_repo │ bulk_save → candle_3343617 (token=3343617): inserting 15000 candles

16:47:40 │ INFO │ app.application.service.historical_service │ [PERF] Chunk 68/69: api=1193ms | transform=66ms | db_write=320ms | rows=15000

16:47:42 │ INFO │ app.infrastructure.postgres.candle_repo │ bulk_save → candle_3343617 (token=3343617): inserting 11625 candles

16:47:42 │ INFO │ app.application.service.historical_service │ [PERF] Chunk 69/69: api=1112ms | transform=127ms | db_write=245ms | rows=11625

16:47:42 │ INFO │ app.application.service.historical_service │ [SUMMARY] 3343617 — api=52.1s (74%) | transform=4.0s (6%) | db_write=13.9s (20%) | total_rows=671002

16:47:42 │ INFO │ app.application.service.historical_service │ ✓ 3343617 done — 671002 candles saved

16:47:42 │ INFO │ app.application.service.historical_service │ [1/1] took 94.9s | Elapsed: 1m 34s | ETA: 0s | Remaining: 0 instruments

16:47:43 │ INFO │ app.application.service.historical_service │ ✓ Batch complete — 1 instruments in 1m 35s

16:47:43 │ INFO │ app.application.service.historical_service │ ✓ Step 3/3 — Fetch complete (job_group_id=774f5580-1b7e-4dc4-bb7a-dabd2b39b5f8)

What i am trying to do is to store these logs in a seperate file or table whichever is good

Hello
I am not expert in db so maybe it's possible i am wrong in somewhere.
Here's my situation
I have created db where there's a table which contain financial instrument minute historical data like this
candle_data (single table)

├── instrument_token (FK → instruments)

├── timestamp

├── interval

├── open, high, low, close, volume

└── PK: (instrument_token, timestamp, interval)
I am attaching my current db picture for refrence also

This is ther current db which i am about to convert

Now, problem occur when i am storing 100+ instruments data into candle_data table by dump all instrument data into a single table gives me huge retireval time during calculation
Because i need this historical data for calculation purpose i am using these queries "WHERE instrument_token = ?" like this and it has to filter through all the instruments
so, i discuss this scenerio with my collegue and he suggest me to make a architecure like this

this is the suggested architecture

He's telling me to make a seperate candle_data table for each instruments.
and make it dynamic i never did something like this before so what should be my approach has to be to tackle this situation.

if my expalnation is not clear to someone due to my poor knowledge of eng & dbms
i apolgise in advance,
i want to discuss this with someone

EDIT :- After taking suggestion or discussing i get into a final conculsion i will improve my indexing for current system if my db row's will increaseas then i will time-series DB like timescalse DB
Thanks you everyone