Hi all,
Self taught user here so please excuse if I am using alternative or inefficient methods - willing to learn if you can help
I use Power Query to gather management information, which is then presented (currently via Excel but hopefully soon with Power BI).
The information I extract uses multiple queries, and looks back over 365 days, typically refreshed weekly. The problems this gives me, is that each week, the data is loading (today-365) days of data which slows the loads down, and that the historical data is not held. Is there a way to tell power query to refresh only the most recent data, and to append this to the previous results?
The alternative I am considering is currently getting power query to refresh weekly, using VBA to push that data into MS Access, and then reading from the access tables in order to present in Excel, which seems a bit backwards.
Any help would be appreciated.
Thanks
John
Hi John,
Welcome to our forum!
Power Query in Excel does not have incremental refresh, but Power BI does. When you move to Power BI you'll be able to avoid refreshing ALL data each week.
Mynda
Hi Mynda,
Thanks for your quick reply, can you confirm if this will also allow me to store the historic data that I need?
I'm uncertain as/when we will be able to use PowerBI, is there an alternative way to do this using Power Query at the moment?
Hi John,
The incremental refresh only gets the data you connect the query to. When you refresh a query it doesn't get that same data again, instead it only gets new data. If you remove the connection to the original data then it will be removed from your model. e.g.
If you get data from a folder containing 5 files numbered 1 to 5, and then next month you add another file numbered 6, the incremental refresh will only get file number 6 as it already has the data from files 1 through 5 from when you built the query. If you later remove files 1 through 5 from that folder, and then refresh the query, it will remove the data for files 1 through 5 from the model.
There's no way to perform an incremental refresh in Excel using Power Query. If your final load location for the data is Power Pivot, then you can load that data to Power Pivot directly, thus avoiding Power Query and any refresh performance issues.
Mynda
What about trying this --> https://www.youtube.com/watch?v=cxOH9qsDunA
Will that do incremental refreshing for Excel?
Hi Jim,
That's not incremental refresh in the true sense. The purpose of incremental refresh is to prevent Power Query getting all the data every time you refresh. With this technique it is still getting all the data again, albeit from another table as opposed to other files.
Mynda
