ActiveSync is hardly a new topic on this blog. And once more I’ll be looking into an aspect of it, that I don’t believe I’ve covered in detail before. Let’s start a little easy, and see if we end up with some more knowledge of how to test/debug/play with Exchange ActiveSync. I am making the assumption you are already aware of the basics like setting up devices and installing servers.
Playing the numbers
Microsoft like all other software companies use versioning to differentiate the releases of their products. This applies even if the product name has nothing to do with where in the line of releases you are. Windows 7 is a nice exception, with it actually being version 7 of Windows too. (Now, we’re not counting minor releases of course.) Exchange, Office, and some other products are coordinated, and Exchange 2010 is version 14 – or “wave 14” in Microspeak. For ActiveSync this is an indicator as to what kind of functionality you can expect – for instance certain features might require Exchange 2007 server side, and Windows Mobile 6.0 client side. (Which unsurprisingly means you might have discrepancies between your Exchange server, and your ActiveSync device, but the server side is backwards compatible so this is not an issue as such.)
On the client side things might be a bit different – there are after all a lot of different clients out there. This is from a Windows Phone 6.5.3 (when synced up with Exchange 2010):
For some strange reason you need to establish a partnership before you can read out which version your device supports. Sort of silly if you ask me, but I didn’t find any other key. Maybe it’s intended to reflect the version in use, as negotiated with the server, and not what the device is capable of.
Well, clever me for finding my way through the registry I suppose… It doesn’t offer much value knowing this by itself. But a not uncommon issue I encounter what with Apple, Nokia, and everyone implementing ActiveSync is knowing what is actually implemented. Oh, sure, you can sync. But can you sync tasks? Or text messages? If in doubt ask the vendor which EAS version they support.
Admittedly these numbers aren’t interesting by themselves any which way you look at it. Looking at another registry key under the same hive we find something called “CommandsSupported” which contains a comma separated list. Here’s what my list looks like:
Now I do not know the meaning of all of these commands by heart (Microsoft has them documented on MSDN), but for instance “SmartForward” means the ability to receive a mail with an attachment, and forward it with the attachment intact, without needing to download the attachment to the device and upload to the server first thus saving some traffic on your data plan. These commands are used in the HTTP POST the device communicates to the Exchange Server. More on that later.
Some of the other registry values are fairly self explanatory, and I’m not digging into these, you will be able to figure out some on your own 🙂 While you can usually always (seemingly) modify a value it might be that you’re not really editing it, as some keys may very well be a read-only representation of a setting stored somewhere else. If this is the case you probably do not have easy access to modify the source.
I believe I’ve mentioned before that ActiveSync is implemented as a series of HTTP POST/GET methods and not a wsdl web service. But let’s recap quickly; this means you can in theory play the ActiveSync game through your browser. However you’ll most likely see a title like this in your browser if you try to surf to the address directly:
If you get the 501/505 error it means that ActiveSync is responding, so it’s not worthless but it’s not telling you a lot either. It is one of the first things I check when I’ve configured ActiveSync on a server – because if you’re not getting this response something might be wrong.
Now if what you want is a quick diagnostic or health check of your ActiveSync service you can check out http://www.testexchangeconnectivity.com for Microsoft’s “Exchange Remote Connectivity Analyzer”, and this tool might indicate if you’re on the right track to a working sync solution. This requires that ActiveSync is exposed to the Internet (which usually is the end goal with EAS), and is not provided as a stand-alone utility you can run on your desktop computer on the LAN. As far as I can tell it reports Exchange 2007 RTM as the supported client version (12.0), and while it will work (and provide good help for) Exchange 2010 it’s not updated so if you for instance uncheck “Allow non-provisionable devices” in your ActiveSync policy you’ll get a 403 error even if that is not entirely accurate (it is by design however). But maybe it’s updated by the time you read this, so don’t rely on my little experiment for this one 🙂
If you want to perform the test your self, and get your hands dirty, read on 🙂
Using the browser to surf is so…end-user isn’t it? To get the HTTP 200 return code you want, you have to hand craft the HTTP POST yourself. I would recommend checking out Fiddler from MSFT: http://www.fiddler2.com (There are of course other options out there too if you have other preferences.)
So, what have you got for me dear Exchange
There’s two parts to building this request; the POST string, and the request headers. Let’s look at the string first.
ActiveSync HTTP POST
Your Exchange server is published to the Internet at an address like https://sync.contoso.com. The virtual directory for ActiveSync (and what you might use in a publishing rule on your ISA/TMG Server) is /Microsoft-Server-ActiveSync. The following parameters string along (bad pun intended):
User – The username. (Only the user alias, not the domain part.)
DeviceId – A unique id for the device that is synchronizing.
DeviceType – An id for the device/model you are using, or the ActiveSync client.
Cmd = The command you want to execute
So say I’ve got this same Windows Mobile device as above a complete string could look like this:
ActiveSync Request Headers
If you only provide this POST you’re no further than using the browser – you also need to specify a couple of parameters for the request header:
User-Agent – The user agent of the HTTP client; could be MSFT-PPC/5.2.5300.
Content-Length – A requirement of the POST method and the length of the request. (Set to 0 to have Fiddler calculate it for you.)
Authorization – A base64 encoded string formatted as such (before encoding) domain\username:password.
Content-Type – The MIME type of the content. Should be application/vnd.ms-sync.WBXML.
MS-ASProtocolVersion – The client version of ActiveSync. Set to 14.0 in our example.
If you get a 403 error check that the username&password combo is correctly encoded.
If it works as expected you should see a HTTP /1.1 200 OK response. It seems you’ll get an empty response body if using the DeviceId of a device that has an ActiveSync partnership configured, whereas using a “fake” id there will be a body. If you inspect the response body returned you’re not likely to see much useful though – for two reasons:
– It’s WBXML so you’ll need to decode it to something readable.
– And if you do you’ll find some error since you didn’t supply all the necessary commands for a full sync in your POST. (As a side note the “Sync” command is one of few commands that are accepted with headers only and an empty body.)
But still, we’re getting somewhere as opposed to the 501/505 we saw earlier 🙂
The sync shown above is just a quick connectivity test – it does not explain how an actual device would interact with Exchange. You might say I started in the wrong end, but in troubleshooting you often try the easy things first before you go all in. While the HTTP 501/505 will identify issues DNS errors, SSL trust issues, and general reachability of the server, the HTTP 200 response (or lack thereof) might identify application level issues.
Being Exchange ActiveSync
So, let’s say I emulate what a clean device would do when going through the configuration wizard. The first part would actually be the AutoDiscover, but this is optional and on most devices you can skip it if you like. (I also don’t consider it a “core” ActiveSync feature.) The next step is pulling down some info from the server through HTTP OPTIONS as the terminology goes. Execute this one (remember correct credentials for this request too):
OPTIONS is one of the things that the Exchange Remote Connectivity Analyzer tests, and it’s not entirely uncommon for this to fail in some scenarios. You don’t configure this on your Exchange Server, and it’s not likely you did something wrong installing your Client Access Server. But some firewalls with tightened security might block the OPTIONS keyword/method. ActiveSync clients may or may not work properly if OPTIONS is blocked (implementation specific). A symptom might be that you’re not able to use all the features your Exchange version provides.
The next thing you would do is do the initial synchronization job. This is done by issuing the “FolderSync” command. And this is where things start to get interesting/challenging/frustrating (strike out the non-applicable ones) 🙂
Do you remember the first sync
There are two accepted methods for stringing together the URI used in the POST. The one we saw above, and a base64 encoded variant – so it would be something like http://sync.contoso.com/Microsoft-Server-Activesync/?abc1234
Now if it was as easy as doing a base64 of the ?User=andreas… variant above you could just use an encoder and figure it out. But it’s not that easy. For instance the server version is also encoded into the URI, and unless you happen to be really good at bytes versus strings you’re not going to be able to do the math without utilities. The good thing? You don’t have to do this. Both methods – plain-text and base64 are accepted. The last one uses a few bytes less and can probably be considered more efficient with regards to bandwidth, but personally I don’t see this as an issue. As far as I can tell Windows Mobile devices use base64, while Nokias and the iPhone use plain-text. (This is only an observation for a few select models and might not reflect the implementation for all of these devices.)
Next challenge? At this point you have to provide a request body, and not just an empty header. As so many other things coming to interacting with web interfaces this is done through writing down some xml. The xml required for the initial synchronization would be similar to this:
<?xml version=”1.0″ encoding=”utf-8″?>
Looks easy enough. Oh, wait, that’s right – there’s an additional step. You need to compact it to binary xml – or wbxml as the formal name is. Which means something like “32 00 12 …”. This is also a task you’re not likely to do without a utility, but although you probably cannot convert in your head you can convert with a hex editor if you are so inclined. Just remember that you cannot enter a string of bytes if you’re using Fiddler, and you have to enter a text representation of the bytes. I’ll admit that at this point I ran into problems using Fiddler for this very reason. These bytes will not encode properly in Fiddler and I had to write up some C# code to do it right. (And while this is relevant I’m not going into the details right now as to not further confuse matters.) Just trust me when I say the request body looks like this:
(If you are interested in the byte sequence – here goes: 03 01 6a 00 00 07 56 52 03 30 00 01 01.)
This time around you must include the following headers:
– Content-Type: application/vnd.ms-sync.wbxml
– User-Agent: xyz
– MS-ASProtocolVersion (see numbers above)
– X-MS-PolicyKey (see below for explanation)
So what should you expect as the response? Well, this would be dependent on which Exchange server you happen to be querying. There are currently three Exchange releases that support ActiveSync – Exchange 2003, Exchange 2007 and Exchange 2010. (Exchange 2003 requires Service Pack 2 for DirectPush, but supports EAS in RTM. Not that I know of many 2003 deployments without SP2…) Exchange 2003 provided a rather lackluster selection of policies that you could implement. Exchange 2007 improved on this and also introduced the concept of non-provisionable devices meaning devices that are not able to implement the desired policies. With Service Pack 1 for Exchange 2007 came the requirement that you could not activate ActiveSync for a user without defining a policy (in RTM this was optional). The policy could be “empty” in the sense that you didn’t actually set any policies, and I think you might be able to “hack” your way around it with PowerShell, but proper behavior is to have a policy. Exchange 2010 doesn’t really add much in the policy department as compared to Exchange 2007, and works similar for the scenario we are playing out.
So what does this boil down to? Well, as I said I’m having problems building the request properly with Fiddler, but we can inspect the traffic with Fiddler. After attempting this FolderSync Exchange now wants to know if you are able to do provisioning, and will want you to actually do the provisioning before you are allowed to do the FolderSync. I have not tested this syncing against Exchange 2003, and it might behave differently. The results I got (working against an Exchange 2010 server) were the following:
Request Header: MS-ASProtocolVersion: 12.0 – X-MS-PolicyKey not present
Request Header: MS-ASProtocolVersion: 14.0 – X-MS-PolicyKey not present
For this I get a HTTP 200 so I have to check the HexView of the response to learn the details
I see “141” meaning:
“A policy is in place but the device is not provisionable.”
There was no psychic guessing involved – I just spotted the numbers and deduced it might be the values referenced in this table:
I reject your policies, and substitute my own
Here’s where it get’s funny. The next thing is that there are some requests and responses going back and forth asking for the provisioning xml. The device then tries another “FolderSync” after it has pulled down the policies. If you have disabled the “allow non-provisionable devices” setting Exchange will say “hold on there boy, have you actually implemented the policies I sent you?” The device will hopefully say yes, and the command will be processed server side. (There are some details I’ve left out – like some simple “keys” generated so you can’t hi-jack the session, but these details aren’t relevant for the bigger picture.) This is entirely an issue of trust, and Exchange cannot actually check if the policies have been applied. So let me re-phrase this behavior in other words: If I do not like your pesky policies, I can implement a client that will always respond that policies are ok regardless of whether the device supports them or not. I can ignore them completely!
This means that if you really want to make sure your users are using the power-on-password feature you cannot do this with Exchange by itself. I’m definitely repeating myself here trying to bring a message across – but you cannot properly implement a strategy for handling mobile devices without having two additional things:
– An MDM solution that will handle management of devices. This includes policy enforcement.
– An ISA/TMG server, ISAPI filter or something similar that will inspect what kind of devices are trying to synchronize the data from the Exchange server.
But back to the topic at hand 🙂
To do this I have to issue a HTTP POST with the “Provision” command using a value of 0 for X-MS-PolicyKey. Exchange will respond with the provisioning xml, and a new temporary PolicyKey. My client will then apply the proper actions, and after this is done I report back to the server with the temporary key and a value of 1 as status indicating I have implemented these policies. I will then be issued the final PolicyKey which I have to include in subsequent operations. (If you change the policies this triggers a new provisioning/PolicyKey cycle on the device.) And yes; I have checked that I can get this working 🙂 (I did a lot of manual byte manipulations as I have not yet written proper parsers to automate things.)
What you get in return from the server after we have played the policy charade will still be in the user-friendly wbxml format, but you will be able to recognize item types you’ve got in your Outlook if you inspect the response. The names will be localized to your chosen language – not sure which variable controls this server side, but that’s an Exchange thing.
Going further from here you can use other commands too like sending mail, etc. While that is a nice thing, it is a bit further than I planned at the moment since the intention of this article was to provide some background for a deep dive troubleshooting session if you ever run into a tricky ActiveSync issue. It also works, (in my opinion), as a primer for understanding the ActiveSync protocol, and not just the configuration & setup bits.
A few tips if you want to play with this on your own:
– WireShark (www.wireshark.org) can parse wbxml if you want to sniff the traffic.
– When learning/debugging it’s much easier if you run the traffic over HTTP rather than HTTPS. Depending on your Exchange server this might be disabled by default and have to be specifically enabled. (Do remember to switch it back on after testing.)
If you want more detail than I’m ever going to be able to supply check out the MSDN library:
It might not be what you’re likely to read as bedtime stories, but it does provide pretty much everything MSFT will release to the general public for the time being. You’ll be interested in the ones starting with “MS-AS”.
I know I indicated a couple of paragraphs ago I resorted to using Fiddler only for inspecting the traffic, and actually creating the requests in code. And you might be feeling that leaves some of this as a theoretical exercise since I didn’t provide an app. Theory will only get you so far, and actually being able to test it proper provides a greater value for most people. I do apologize for that, but I felt that mixing in code in this article would possibly make it less understandable/readable and scare away the non-programmers. I am working on this separately however, and I’m looking to release some sort of utility to help you along the way.
Phew, that was a longer run than expected – so till next time; get to it and start “breaking” your ActiveSync 🙂