Click on the Category button to get more articles regarding that product.
Exchange 2010/2013/2016 Migration,
problem after DNS-pointing to 2016 structure with some users Outlook.exe
When you thought Kerberos Bloating is way back 2012 it returns. And after some research it is still all over the place. It does affect on premise Solutions as well as cloud solution like ADFS, AZURE etc.
Error:
This error (HTTP 400 Bad Request) means that Internet Explorer was able to connect to the web server, but the webpage could not be found because of a problem with the address.
Reason:
Active Directory User being in member of too many Active Directory groups. Kerberos Ticket Bloating.
Solution:
You can modify a Parameter on the 2010 CAS to allow larger Kerberos Packets to be used for Authentication to Webservices. This may be also valid for other problems where you Authenticator to a Web server solution with Kerberos (Active Directory) as sample: Ticket Systems, Intranet Solutions, SharePoint, Security Appliance etc.
Pro Keyfacts:
You can test the effects by opening the Autodiscover URL in a web browser. Don’t handle too much outlook.exe not opening (Because Autodiscover does not work at that moment you want be able)
MS says on the CAS 2010 only. HPE Services once had a KB which said DC and CAS. (Maybe older DC’s that time)
Problem:
We have read/heard with number like user ADS-User-Object in 200+ ADS-groups. So at that point we dropped further research and did think this does not affect the customer because he had max 130 Groups a user was in.
But one employee was affected where one user was in only 83 groups and the second user was in 127 groups. There seem to be other Kerberos info which adds to that and hits some limits when the Kerberos packet is proxy back from Exchange 2016 to 2010 and then to the Domain Controller.
You can count the memberships with a 3-liner in PS:
$user=get-aduser m.butsch
$token=(get-aduser $user -Properties tokengroups).tokengroups
$token.count
At the Point where most is setup and you move the Autodiscover SCP DNS to the Exchange 2016 some people are:
- Unable to open Outlook.exe (User still on 2010, Just SCP Point new to 2016)
- Unable to Create New Outlook.exe Profile
- Unable Reach the Autodiscover URL in Web Browser when the DNS Record for that points to the 2016 structure. (https://autodiscover.yourdomain.ch/autodiscover/autodiscover.xml)
The key fact is that you can Test and DEBUG this with by just opening the URL in a web browser. So you don’t have to handle around with outlook.exe /rpcdiag. Because you can’t open Outlook.exe you are also unable to test Autodiscover with Right on Outlook symbol.
This may be a Pitfall if you had Kerberos Authentication in place and because of that reason FOCUS too much in that direction. If you want to take over Kerberos Authentication from 2010/2016 you may have to build back and then on 2016 build it up again.
This is how the effect shows up on a Client with Outlook 2016.
“Die systemresourcen sind sehr niedrig. Schliessen Sie einige Fenster”
This is how the Outlook Profiles Look after some debug sessions if the effect is there:
To test if the effect is there:
As user > If you open the Autodiscover URL in different Browser you get Error 400
Google Chrome:
Internet Explorer 11 (Because mostly people set PROXY and EXCEPTIONS there and then other Browser import it from that) > which has to be working for Exchange 2013/2016 with Web Proxy active. (you have to exclude all Exchange 2010/2013/2016/2019 FQDN from proxy)
On the Exchange 2016 Server Logfiles you see following error:
“C:\Program Files\Microsoft\Exchange Server\V15\Logging\HttpProxy\Autodiscover\*.log”
Check the SIZE of the logfiles growing as soon as you hand over the DNS for Auto discover from 2010 to 2016 Servers.
Search for text: The remote server returned an error: (400) Bad Request.
2021-06-22T10:28:44.962Z,f2c25044-6223-41d6-9737-da6f010f1ffe,15,1,2242,10,{49184D7A-04D7-47BD-977C-A0DE7BC9AA8B},Autodiscover,autodiscover.fda.ch,/Autodiscover/Autodiscover.xml,,Negotiate,true,fda\u1234,fda.ch, Smtp~Linsi.vonn@fda.ch,Microsoft Office/16.0 (Windows NT 10.0; Microsoft Outlook 16.0.5149; Pro),172.30.46.211,fdaEXC7,400,400,,POST,Proxy,fdacas2.fda.ch,14.03.0123.000,IntraForest,AnchorMailboxHeader-SMTP,,,,353,346,,,9,1,,0,1;,1,,0,1,,0,14,0,1,0,0,0,0,1,0,0,0,5,0,1,3,3,12,14,,,,BeginRequest=2021-06-22T10:28:44.947Z;CorrelationID=<empty>; ProxyState-Run=None;AccountForestGuard_fda.ch=1;DownLevelTargetRandomHashing=0/3;ClientAccessServer=fdaCAS2.fda.ch; ResolveCasLatency=0;FEAuth=BEVersion-1937997947;ProxyToDownLevel=True;RoutingEntry=DatabaseGuid:81fdd93e-6b0a-49f0-ae6b-c41619e3ebad%40fda.ch%40fda.ch Server:fdaEXC4.fda.ch+1937997947@0;BeginGetRequestStream=2021-06-22T10:28:44.960Z;OnRequestStreamReady=2021-06-22T10:28:44.960Z;BeginGetResponse=2021-06-22T10:28:44.961Z;OnResponseReady=2021-06-22T10:28:44.961Z;EndGetResponse=2021-06-22T10:28:44.961Z;ProxyState-Complete=ProxyResponseData;SharedCacheGuard=0;EndRequest=2021-06-22T10:28:44.962Z;I32:ADS.C[fdaDCW2]=1;F:ADS.AL[fdaDCW2]=0.9151757;I32:ATE.C[fdaDCW2.fda.ch]=1;F:ATE.AL[fdaDCW2.fda.ch]=0, WebExceptionStatus=ProtocolError;ResponseStatusCode=400;WebException=System.Net.WebException: The remote server returned an error: (400) Bad Request. at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<>c__DisplayClass197_0.<OnResponseReady>b__0();,,|RoutingDB:81fdd93e-6b0a-49f0-ae6b-c41619e3ebad,,,CafeV1
On Domain Controllers you may see following just at that time you open the Autodiscover URL in the browser of the client:
You may see following Error from the Exchange 2010 CAS Server on one of your Domain Controller. Check Events under Security for “Event 4769, Audit Failure”
| A Kerberos service ticket was requested. Account Information: Account Name: FDACAS2$@FDA.CH Account Domain: FDA.CH Logon GUID: {00000000-0000-0000-0000-000000000000}
Service Information: Service Name: FDAcas2$@FDA.CH Service ID: NULL SID Network Information: Client Address: ::ffff:172.30.46.134 Client Port: 54554 Additional Information: Ticket Options: 0x40810000 Ticket Encryption Type: 0xFFFFFFFF Failure Code: 0x12 Transited Services: – |
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> <System> <Provider Name=”Microsoft-Windows-Security-Auditing” Guid=”{54849625-5478-4994-A5BA-3E3B0328C30D}” /> <EventID>4769</EventID> <Version>0</Version> <Level>0</Level> <Task>14337</Task> <Opcode>0</Opcode> <Keywords>0x8010000000000000</Keywords> <TimeCreated SystemTime=”2021-06-22T09:43:13.941534700Z” /> <EventRecordID>1163245364</EventRecordID> <Correlation /> <Execution ProcessID=”484″ ThreadID=”1188″ /> <Channel>Security</Channel> <Computer>fdaDCW2.fda.ch</Computer> <Security /> </System> <EventData> <Data Name=”TargetUserName”>fdaCAS2$@fda.CH</Data> <Data Name=”TargetDomainName”>fda.CH</Data> <Data Name=”ServiceName”>fdacas2$@fda.CH</Data> <Data Name=”ServiceSid”>S-1-0-0</Data> <Data Name=”TicketOptions”>0x40810000</Data> <Data Name=”TicketEncryptionType”>0xffffffff</Data> <Data Name=”IpAddress”>::ffff:172.30.46.134</Data> <Data Name=”IpPort”>19080</Data> <Data Name=”Status”>0x12</Data> <Data Name=”LogonGuid”>{00000000-0000-0000-0000-000000000000}</Data> <Data Name=”TransmittedServices”>-</Data> </EventData> </Event> |
Solution:
Depending on the source where this was offered you may have to adapt that on the:
Exchange 2010 CAS (IIS)
But I found some articles from HPE-IT-Services where it says also on the DC. DO not change it on the DC if the change on the CAS works.
Remember if you have as example several CAS behind a load balancer that the effect is backwards from the Exchange 2016 to the 2010. There is only A little of that process which will go over the front of the Load Balancer (Like KEMP or F5). So you have to patch all CAS.
If am not aware if CAS Server you EXCLUDED from CAS-Serving Service are also affected by this or not.
https://docs.microsoft.com/en-US/exchange/troubleshoot/client-connectivity/400-bad-request
Microsoft says:
On every Exchange 2010 CAS, locate the following subkey:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\HTTP\Parameters
Under this subkey, increase the MaxFieldLength and MaxRequestBytes entries by using the values in the following table.
Value name Value type Value data Value base
MaxFieldLength DWORD 65536 Decimal
MaxRequestBytes DWORD 65536 Decimal
To check if it’s working:
Open up https://autodiscover.yourdomain.ch/autodiscover/autodiscover.xml
A Credentials POPUP is fine if not also.
But you have to see the XML File and Error 600 then all is fine.
Find Autodiscover endpoints by using SCP lookup in Exchange | Microsoft Docs
Powershell to check the group membership of all ADS-user to be run on your DC.
Makes a Text Lofile: user_groups.txt
| # V1.0, 22.06.2021, M. Butsch, www.butsch.chstart-transcript -path user_groups.txt$Users = Get-ADUser -Filter * -Properties Name, GivenName, SurName, SamAccountName, UserPrincipalName, MemberOf, Enabled -ResultSetSize $NullForeach($User in $users){
$MA=get-aduser $User $token=(get-aduser $MA -Properties tokengroups).tokengroups $MATOKEN=$token.count Write-Host $MA.SamAccountName’;’$MA.name’;’$MATOKEN } stop-transcript |