Remove the list view threshold (5000 by default)
This limit has always been a bit laughable, and is even more so as we develop more client side applications. In SharePoint 2007 we didn't have this limit and were allowed to make our own mistakes. Now that hardware is so much more powerful, we need this limit removed so that we can build enterprise-class applications.
We are continuing to make our large list experiences better, please keep the feedback coming.
Spring 2018 update:
- We now support being able to manually add indexes to lists of any size (increased from lists up to 20,000 items previously).
- Starting with the February release of the Office 365 Excel client, you will be able to export your full list instead of getting cut off part of the way through.
What we are working on now:
- Predictive indexing will start to work for lists larger than 20,000 items so your views will automatically cause the right indexes to be added to your lists.
In our backlog:
- Being able to index/sort/filter by lookup column types (like person, lookup or managed metadata columns) without being throttled.
- Making sure that our REST APIs support querying in ways that will guarantee that the call will not be throttled.
For a general update on large list capabiltiies, the video on myignite.microsoft.com/videos/53861 (focusing on large lists at 42 minutes and 25 seconds) describes some of the changes that we delivered back in the second half of 2017:
- Modern UI now has a lot of support for adding indexes to large lists and libraries on the fly, reducing the number of throttling errors experienced by our users, and some new UI for browsing through items in large lists with our paging model
- SharePoint runs predictive indexing jobs to automatically add indexes as lists get larger based on your view definitions, and updates these indexes when you add/update your views
Looking forward to receiving more of your feedback.
It is ridiculious that these limites are still place. No real progress. Why not just come with a real Cloud enterprise grade file storage system that canbe synch'd to computers and mapped drives for a hybrid on-premise? Currently, Microsoft has forced me to find an alternate system.
So this limitation has become a big problem for me recently, even with my fields all indexed properly.
I am using RenderListDataAsStream for ordering, filtering and paging all together.
But it seems like the paged queries don't look at all items, it looks at chunks of 5k items, and returns matches from that chunk.
So, I might have two pages with no results, before the third page gets matches.
This is something the modern view for a List does seamlessly, to load a full page of sorted, filtered data, but I can't figure out how it was done.
When bypassing normal methods like the Rest api or ms-graph, to instead use the CAML streaming api, and encounter the same limitation. I might as well just use AJAX on its own.
It is really important to show any alert or send information mails to the owners of the site before any list view reaches its threshold. This way, users can try various workarounds and take precautions around indexing and manage large list related problems proactively.
This needs to be made a top priority. We need to be able to override this and make exceptions. Try working for a big global company and have libraries constrained like this. Business users can't get their data out on their own. Everything is blocked! Can't use file explorer and can't use SharePoint Designer.
Microsoft, is there any update on this? It's 2020 and still no solution or update on this subject :(
I am hitting a brick wall with this as well. Really need to be able to download data in bigger chunks using powershell. EX: Get-PnPListItem The pagesize parameter just doesn't work for me.
I need be able to pull 20,000 records at a time. Only about 6 text columns, entire payload is only a few MB.
This really needs to be addressed! Moving my site back on-prem to accommodate this limitation.
tldr; I copy lookup to singlelinetext column and sort/filter on that.
To get around lookups not indexing, I copy the value of the lookup into a singlelinetextbox (using PowerAutomate) and then you can sort, filter etc with that singlelinetext column. Further, you can hide the lookups in your views and in the forms make the singlelinetext column hidden. As for when the lookup is updated, you can have another flow that runs and updates records using that lookup (I use PowerShell to silent update the item to bypass version history, etc) This is far from an ideal solution, but I get passed the 5k limit and my client(s) are happy. Its the year 2020 microsoft, while we were promised hoverboards by TV this year, we accept it not happening, but at least let us settle for indexed lookups. :-) I understand this is a complex (probably the most complex) feature you guys are working on for SharePoint, but come, bring it, and make SharePoint Great Again!
Or let us increase thresholds :-)
Alex R commented
Microsoft Documentation states a library will support up to 30 million items.
But after 100,000 items, you can no longer manage permission inheritance!
So in reality, SPO only fully supports libraries with up to 100,000 items.........
Tanveer Hirani commented
I have designed a library for sharing with internal staff and customers (via folders) for individual property/buildings. The library is expected to have more than 50,000 documents added each year by contractors! Even though I have created almost 20 different views, I really need a view with more than 5,000 documents! Because of the nature of our business. It is critical that this limit is removed for my solution and design to work which is around integrating 10 teams within my department and move away from network/windows explorer based filing/document management system.
Another issue that I have is that I cannot index multiple values Managed Metadata columns which makes my document library kinda useless. Please try improve or remove this limit.
Brendan Murphy commented
Just hit this issue today "- Being able to index/sort/filter by lookup column types (like person, lookup or managed metadata columns) without being throttled." My solution is dead in the water because of this. This is really ridiculous. Now I have to come up with some convoluted way to move items from one list to another and keep the history so that my lookup column will work. I've been a big SP fan, but since moving to SPO, I'm becoming less and less of one.
Rajendra Katta commented
Microsoft - Migration of large lists from SharePoint On-premise to SharePoint online is getting horrible, Some of the lists have 100k items migrating them into SharePoint Online is useless after migration. Please increase the Limit ASAP. The clients are absolutely disappointed with Threshold limit.
Microsoft - please do something about this limit ASAP! SharePoint lists are useless for business applications where you have more than 10 users that add items on a daily basis.
Maamar Bouzada commented
We need to extend the limitation of 5000 items view to bigger .
What is happening to this? This problem needs an urgent resolution. Almost every SP site we have will exceed 5000 files. It is severely impacting our ability to use SP at the enterprise level.
Marco Garibay commented
One year since the last update and nothing yet?
Speaking for our team, we just want to use sharepoint as a file repository and so far it's been a nightmare.
I'd like to see a sharepoint container that behaves exactly like a onedrive, with zero limit of file size transfer.
This is important as we use onedrive to store video files which go above 5GB
Please remove this block.
Please increase the number. It is urgent!