Hello Randhir,
Just saying 'this is not fast enough' is usually not helpful. Execution plans from the IQ side might show what's missing for you to tune up your scenario.
Anyhow, assuming your issue is happening only when you use SDA I have a few observations on your description:
- Retrieving 13M records from IQ does not seem to be a good idea. sending those over the wire will take time for sure. Maybe you're not aggregating data at the source and obviously joining such a huge amount of data can take time. Thus, aggregation and filtering at the source (if possible) will surely help. Indexing the join columns should also help. More info on IQ indexing/tuning here.
- Data preview usually gets the TOP 1000 results, so there's a clear difference in materialized data volume.
Other than that I guess the iq space would have more inputs on that. Thus, I'm moving this discussion there.
BRs,
Lucas de Oliveira