Power query reading sharepoint files takes 20mins to refresh.

Hi Thomas,

Thank you for the forum question.

It sounds like you've put a lot of effort into setting up your Power Query, and it's frustrating to see it slow down so much after switching to SharePoint. Here are some potential reasons and solutions for the slow performance:

Potential Reasons for Slow Performance
Data Source Connection: SharePoint connectors can be slower compared to local or OneDrive connections due to network latency and the way data is accessed.
Complex Data Transformations: If your query involves multiple steps and transformations, it can slow down the refresh process.
Large Data Volume: Handling a large number of files and rows can significantly impact performance.
Tips to Improve Performance
Filter Early: Apply filters as early as possible in your query to reduce the amount of data being processed.
Use Table.Buffer: Buffering tables can help improve performance by reducing the number of times data is read from the source.
Optimize Data Types: Ensure that you are using the correct data types for your columns to avoid unnecessary conversions.
Disable Auto Date/Time: Turn off the auto date/time feature in Power Query settings to speed up processing.
Use SharePoint.Contents: Instead of using the default SharePoint folder connection, try using SharePoint.Contents which can sometimes be faster.
Steps to Implement Changes
Filter Early:

let
Source = SharePoint.Files("https://yourcompany.sharepoint.com/sites/yoursite"),
FilteredFiles = Table.SelectRows(Source, each Text.EndsWith([Name], ".xlsx")),
// Apply other transformations here
in
FilteredFiles
Use Table.Buffer:

let
Source = SharePoint.Files("https://yourcompany.sharepoint.com/sites/yoursite"),
BufferedSource = Table.Buffer(Source),
// Apply other transformations here
in
BufferedSource
Disable Auto Date/Time:

Go to File > Options and Settings > Options.
Under Current File > Data Load, uncheck Auto Date/Time.
By implementing these optimizations, you should see an improvement in the performance of your Power Query. If the issue persists, it might be worth exploring other data storage solutions or consulting with your IT department for further assistance

Kind regards

Jens Bonde
Microsoft Office Specialist Trainer

Tel: 0207 987 3777
STL - https://www.stl-training.co.uk
98%+ recommend us

London's leader with UK wide delivery in Microsoft Office training and management training to global brands, FTSE 100, SME's and the public sector