Improved API feedback around throttling
Currently working with the API's for drive, graph, sharepoint are very difficult due to the throttling that can be encountered.
Each api offers different guidance to avoid throttling. Example are:
1) use a special user-agent header
2) respect the retry-after header
Speaking with microsoft representatives they instruct us to perform major api processing in off peak hours. Additionally tenant's can have hard locks placed on them.
This request is to get better information around tenant usage returned in the API's.
1) If a tenant is hard locked, tell us in a distinct message so that don't just hammer the requests over and over again.
2) Give us feedback in our api calls regarding the health of a tenant. If we are approaching a threshold limit, give us some type of header we can use to instruct us to slow down our requests.
ruel loehr
shared this idea
We are looking in to doing work on these areas, but we cannot provider an exact timelines as this point.
3 comments
-
[Deleted User]
commented
The `x-sharepointhealthscore` response header is an indicator for how busy SharePoint is. Anything above 6 usually means very high load (but you could probably extrapolate that from response delays too).
-
Anonymous
commented
Hi folks,
Can confirm that this issue (throttling) is affecting our customers.
Some of them would love to backup/restore to OneDrive but throttling is killing the performance and causes issues with restore. It would be amazing if user could control the throttling option (and disable it if required, maybe set a limit for a specific application).
Thank you in advance and hope to see this feature request implemented. Cheers! Dima P.
-
Michael R
commented
Hello Product Team,
I can confirm this is a really crucial suggestion. We're observing the same API calls throttling related issues within Veeam Agent for Microsoft Windows recovery operations when after a while regardless possible throttling handling steps the recovery fails due to the throttling of API calls:
[24.10.2019 01:55:08] < 348> stg| [debug] Waiting 124984 MS before next OneDrive API call
[24.10.2019 01:55:08] < 1888> stg| WARN|OneDrive API call failed. Number of attempts: [1]. Should be retried: [1]
[24.10.2019 01:55:08] < 1888> stg| >> |OneDrive error: 'The request has been throttled
[24.10.2019 01:55:08] < 1888> stg| >> |Inner error: throttledRequest'
[24.10.2019 01:55:08] < 1888> stg| >> |--tr:Request URI: '/drive/root:/VeeamBackup/3edb884f-081c-43f4-9131-d231d678f53e/BlockStore/%7Bc2653075-ec25-4934-817d-b61197996e8a%7D/eb/ebcff0fbfcd491e74c6da41a36f0d24f:/content'
[24.10.2019 01:55:08] < 1888> stg| >> |--tr:Request Method: 'GET'
[24.10.2019 01:55:08] < 1888> stg| >> |--tr:Response body: '{"error":{"innerError":{"code":"throttledRequest"},"code":"activityLimitReached","message":"The request has been throttled","retryAfterSeconds":120}}'
[24.10.2019 01:55:08] < 1888> stg| >> |--tr:Reason phrase: 'Too Many Requests'
[24.10.2019 01:55:08] < 1888> stg| >> |--tr:HTTP code: '429'The full log is available at the following link if needed -https://storage.veeam.com/tmp/File_03822547_e40b7209f2.zip
Let me know if any details are needed to help for further possible improvements.
