Hi All,
In Mynda's PQ Course 2.05 on From Folders she advises the use of the Transform Sample File to carry out the necessary transformations. I need to reference the File Name so I'm wondering what the negative impact is for not using the "Transform Sample File" query and jumping straight to the combined query?
Will added files to this folder still have their data incorporated after a refresh? Or is there something else that is impacted?
Hi Rod,
It's ok to make transformations in the combined query, except where those transformations must be done before combining the files. e.g. unpivotting data must be done prior to combining.
Hope that clarifies things.
Mynda
Ok, So I have actually done the unpivot step in the combined Query. This is because I've used part of the filename as data in the query. Should I go back and redo the Query using the Sample file and essentially get 1 row of unpivoted, transformed data with many columns?
Thanks Rod
Actually, I'm struggling to see how I would use the Transform Sample File query because this query doesn't contain the filename. So I'm a little confused as to how I could have done the required Transformations without using the Combined Query. Can you provide some suggestion/s? BTW the Combined Query works, so it's really about whether I'm going to suffer performance issues. It looks like I'll have 1000,000 Rows.
Whether you make the changes in the sample or the final query won't make any performance differences. If you're able to make the changes you need in the final query, then do it there. If you aren't able to, then it means you probably should have made the changes in the sample query.
Hi Mynda,
Thank you that's good to know.
I'm loving the courses BTW.
Kind Regards
Rod
