I have a dataset with 4.5million rows and need to pull all rows for selected clients.
My thought was to use a second table (e.g. client Id) to provide a means to filter output to selected client records so that I can then drill down to the full record candidate list.
Problem is, when adding the client id to the rows section, it will add a (blank) row to cover all the clients not in the candidate list which then means the drill down tries to show 4.5 million rows. Trying to filter out the (blanks) entry errors that you can’t show details on a filtered column/row.
any thoughts about how to get around this, or alternatives to provide sub-sets from a larger set?
Cheers,
Steve
Hi Steve,
Welcome to our forum!
There are a couple of ways you can do this but it depends on your needs. When you say 'selected clients' is this list likely to change regularly e.g. like a user might use a Slicer to chop and change what filters they apply, or is this something more permanent that you'd only change periodically or even never?
If the latter, then you can filter out the data you don't want in Power Query before loading it to your data model/Power Pivot. This would be far more efficient for your model. You can still change the filters applied in Power Query, even by modifying a list in the worksheet, but refreshing it will be slower.
Mynda
Hi Mynda,
it will be a regular thing. We use one pivot to identify clients that meet particular criteria e.g. count of clients with “high” transaction counts by month to look for areas of interest. but since that would only show details for those instances (month), we then seek to retrieve all records for those clients.
There are around 800,000 clients with sub-groups generally being in the 10s of thousands so filtering on that list up-front is a challenge.
the alternative is to put all 42 fields in the rows section and avoid the need to show details but at that size, excel chokes.
i probably need to come at this from a different angle, just locked in on the power pivot at the moment 🙂
