Remove the list view threshold (5000 by default)
This limit has always been a bit laughable, and is even more so as we develop more client side applications. In SharePoint 2007 we didn't have this limit and were allowed to make our own mistakes. Now that hardware is so much more powerful, we need this limit removed so that we can build enterprise-class applications.
We are continuing to make our large list experiences better, please keep the feedback coming.
Spring 2018 update:
- We now support being able to manually add indexes to lists of any size (increased from lists up to 20,000 items previously).
- Starting with the February release of the Office 365 Excel client, you will be able to export your full list instead of getting cut off part of the way through.
What we are working on now:
- Predictive indexing will start to work for lists larger than 20,000 items so your views will automatically cause the right indexes to be added to your lists.
In our backlog:
- Being able to index/sort/filter by lookup column types (like person, lookup or managed metadata columns) without being throttled.
- Making sure that our REST APIs support querying in ways that will guarantee that the call will not be throttled.
For a general update on large list capabiltiies, the video on myignite.microsoft.com/videos/53861 (focusing on large lists at 42 minutes and 25 seconds) describes some of the changes that we delivered back in the second half of 2017:
- Modern UI now has a lot of support for adding indexes to large lists and libraries on the fly, reducing the number of throttling errors experienced by our users, and some new UI for browsing through items in large lists with our paging model
- SharePoint runs predictive indexing jobs to automatically add indexes as lists get larger based on your view definitions, and updates these indexes when you add/update your views
Looking forward to receiving more of your feedback.
Rob Bowman commented
So, after all this build up we get a simple - sorry we can't get this done. Especially when MS continues to advertise as a Content and Document Management Leader. Greatly disappointed.
"(noting that SQL will never scan more than 5k rows"
Can you explain why SQL will never scan more than 5k rows?
Wake up, Microsoft, it's 2018
Please, do something about this huge problem. Why not a whole new system for filesharing?
Terill Derick commented
Hey, Microsoft, what's going on, help us here...
This limit is terrible and the fact that once you are over the limit you can't index or view the data to reduce the number is crazy. We are exploring other alternatives, so we can migrate away from SP, even though we have been customers since SP2007.
DO SOMETHING N O W!
Please, we want to move all our files to SharePoint Online, but have about 10 sites, some with 30-50 000 files on it. What should we do to prepaire us. We just want a simple file storage?
No, it's not reasonable! We need help, NOW
DO SOMETHING NOW!!!!
This is not acceptable to our organization, which has over 20,000 files in each work area, already divided. If you have problems with SQL, do something about it now. We pay a lot of money for the service. It's a joke in 2018 ....
Can a feature be added to some queries such as the managed checked out versions so they can bypass the threshold limit? That function is crippled by the threshold limit.
1 major benefit of moving to SharePoint online is the capacity, this is not make sense SharePoint online has such limit.
[Deleted User] commented
The problem with this is, yet again from Microsoft, very poor programming!!
The view queries being passed back to the SQL backend are clearly inefficient. On looking for workarounds for this poxy limit, I found that you cannot us OR in a view filter as the query fetches ALL records back and then tries to filter it out rather than using the OR in the SQL query and only returning the records you require.
Chris Leeworthy commented
This is not useful in my case.
I have just migrated multiple sites from old SharePoint 2007 on premise to SharePoint Online.
The lists are already defined and populated with more than 5000 items so I can't go in and fix them by creating new views or indices because they already break the view limit!
How am I supposed to get these lists working again when any attempt to fix them is blocked?
Gunnar Lied commented
My suggestion is: Create a new and dynamic data model without having all eggs in the same basket (like AllUserData), in parallel to the old one, so everything can scale as you would expect in a modern platform. I'm sure it's painful, but all the current limitations are also painful.
I understand the technical issue and how to overcome it, but not all users are willing to do the leg work up front for several reasons. The biggest reason is probably because they don't initially know their list is going to scale to that size. We are moving from SharePoint 2007, where the LVT didn't exist so how do we migrate content or tell users they have to follow new rules now. MS get it together and get this fixed or make it look like it's fixed, stop passing the buck to the end-user
Any update on this issue? A platform that is built wish such severe limitations is hard to justify... Please help, A loyal 2 decade Microsoft customer.
This issue reminds me of the win8 start button. one step forward two steps back.
Our customers are "thinking about it" too... thinking about leaving the platform. Please for starters at least allow us to index lists that are over the limit without having to delete or migrate data out. Perhaps the underpinnings of the Custom List need to be completely revamped. 5000 is an embarrassing limit we just can't explain to customers.