Remove the list view threshold (5000 by default)
This limit has always been a bit laughable, and is even more so as we develop more client side applications. In SharePoint 2007 we didn't have this limit and were allowed to make our own mistakes. Now that hardware is so much more powerful, we need this limit removed so that we can build enterprise-class applications.
We are continuing to make our large list experiences better, please keep the feedback coming.
Spring 2018 update:
- We now support being able to manually add indexes to lists of any size (increased from lists up to 20,000 items previously).
- Starting with the February release of the Office 365 Excel client, you will be able to export your full list instead of getting cut off part of the way through.
What we are working on now:
- Predictive indexing will start to work for lists larger than 20,000 items so your views will automatically cause the right indexes to be added to your lists.
In our backlog:
- Being able to index/sort/filter by lookup column types (like person, lookup or managed metadata columns) without being throttled.
- Making sure that our REST APIs support querying in ways that will guarantee that the call will not be throttled.
For a general update on large list capabiltiies, the video on myignite.microsoft.com/videos/53861 (focusing on large lists at 42 minutes and 25 seconds) describes some of the changes that we delivered back in the second half of 2017:
- Modern UI now has a lot of support for adding indexes to large lists and libraries on the fly, reducing the number of throttling errors experienced by our users, and some new UI for browsing through items in large lists with our paging model
- SharePoint runs predictive indexing jobs to automatically add indexes as lists get larger based on your view definitions, and updates these indexes when you add/update your views
Looking forward to receiving more of your feedback.
I just migrated some server drives to our SP online site using the official Microsoft SharePoint Migration Tool.
The tool happily migrated 120,000 files to a document library with no errors or warnings. Little did I know that I can't even permission the new library because I have >5000 files!!!
Please fix this limitation ASAP. SharePoint is useless to us with this limitation.
We spent a lot for O365 E3 every year, and this limit makes us think it's not worthed anymore to spend more for only O365 subscription
I can't believe we still can't rename folders with >5000 files due to this issue! BOX has been able to do this for over 10 years! How has there still been no update for over 5 years?
A lot of angry people here for a feature that we've wanted for nearly five years now.
Yusuf Bhayat commented
This issue is making us look really bad to our customers - we can't even present a simple solution as the limit prevents a lookup field from returning results.
Andy Dale commented
How can this problem still exist 2 years after the Admin here claimed it was fixed. We are just trying to have a View grouped by two columns and as soon as we hit 5000 documents, it breaks. All columns are indexed. What is the latest update on this issue. We are using Modern pages.
This problem is making me look really bad within my company. I'm really trying to sold O365 solution as a real collaboration system but I keep hurting myself whenever I encounter there ridicolous limitations.
Do you really think that ANY company that uses a Sharepoint list would EVER stay below 5000 items ?
It is ridiculious that these limites are still place. No real progress. Why not just come with a real Cloud enterprise grade file storage system that canbe synch'd to computers and mapped drives for a hybrid on-premise? Currently, Microsoft has forced me to find an alternate system.
So this limitation has become a big problem for me recently, even with my fields all indexed properly.
I am using RenderListDataAsStream for ordering, filtering and paging all together.
But it seems like the paged queries don't look at all items, it looks at chunks of 5k items, and returns matches from that chunk.
So, I might have two pages with no results, before the third page gets matches.
This is something the modern view for a List does seamlessly, to load a full page of sorted, filtered data, but I can't figure out how it was done.
When bypassing normal methods like the Rest api or ms-graph, to instead use the CAML streaming api, and encounter the same limitation. I might as well just use AJAX on its own.
It is really important to show any alert or send information mails to the owners of the site before any list view reaches its threshold. This way, users can try various workarounds and take precautions around indexing and manage large list related problems proactively.
This needs to be made a top priority. We need to be able to override this and make exceptions. Try working for a big global company and have libraries constrained like this. Business users can't get their data out on their own. Everything is blocked! Can't use file explorer and can't use SharePoint Designer.
Microsoft, is there any update on this? It's 2020 and still no solution or update on this subject :(
I am hitting a brick wall with this as well. Really need to be able to download data in bigger chunks using powershell. EX: Get-PnPListItem The pagesize parameter just doesn't work for me.
I need be able to pull 20,000 records at a time. Only about 6 text columns, entire payload is only a few MB.
This really needs to be addressed! Moving my site back on-prem to accommodate this limitation.
tldr; I copy lookup to singlelinetext column and sort/filter on that.
To get around lookups not indexing, I copy the value of the lookup into a singlelinetextbox (using PowerAutomate) and then you can sort, filter etc with that singlelinetext column. Further, you can hide the lookups in your views and in the forms make the singlelinetext column hidden. As for when the lookup is updated, you can have another flow that runs and updates records using that lookup (I use PowerShell to silent update the item to bypass version history, etc) This is far from an ideal solution, but I get passed the 5k limit and my client(s) are happy. Its the year 2020 microsoft, while we were promised hoverboards by TV this year, we accept it not happening, but at least let us settle for indexed lookups. :-) I understand this is a complex (probably the most complex) feature you guys are working on for SharePoint, but come, bring it, and make SharePoint Great Again!
Or let us increase thresholds :-)
Alex R commented
Microsoft Documentation states a library will support up to 30 million items.
But after 100,000 items, you can no longer manage permission inheritance!
So in reality, SPO only fully supports libraries with up to 100,000 items.........
Tanveer Hirani commented
I have designed a library for sharing with internal staff and customers (via folders) for individual property/buildings. The library is expected to have more than 50,000 documents added each year by contractors! Even though I have created almost 20 different views, I really need a view with more than 5,000 documents! Because of the nature of our business. It is critical that this limit is removed for my solution and design to work which is around integrating 10 teams within my department and move away from network/windows explorer based filing/document management system.
Another issue that I have is that I cannot index multiple values Managed Metadata columns which makes my document library kinda useless. Please try improve or remove this limit.
Brendan Murphy commented
Just hit this issue today "- Being able to index/sort/filter by lookup column types (like person, lookup or managed metadata columns) without being throttled." My solution is dead in the water because of this. This is really ridiculous. Now I have to come up with some convoluted way to move items from one list to another and keep the history so that my lookup column will work. I've been a big SP fan, but since moving to SPO, I'm becoming less and less of one.
Rajendra Katta commented
Microsoft - Migration of large lists from SharePoint On-premise to SharePoint online is getting horrible, Some of the lists have 100k items migrating them into SharePoint Online is useless after migration. Please increase the Limit ASAP. The clients are absolutely disappointed with Threshold limit.