I have been picking on Google lately. In fact, all of my blog posts thus far have focused on Google things. Earlier this year I wrote a blog about Android Auto, Google’s solution for unifying telematic user interfaces (UIs), and in it I mentioned that I am a daily CarPlay driver. So, in the interest of being fair, I thought I would pick on Apple for a bit and take a look under the hood of CarPlay, Apple’s foray into automotive telematics.
Worldwide, 62 different auto manufacturers make over 500 models that support CarPlay. Additionally, 6 after-market radio manufacturers (think Pioneer, Kenwood, Clarion, etc.) support CarPlay. In comparison, 41 auto manufacturers (again, over 500 models – this is an increase since my earlier post) and 19 after-market radio manufacturers support Android Auto. CarPlay runs on iPhone 5 and later. It has been a part of iOS since its arrival (in iOS 7.1), so there is no additional app to download (unlike Android Auto). A driver simply plugs the phone into the car (or wirelessly pairs it if the car supports it) and drives off; a wired connection negates the need for a Bluetooth connection. The toughest thing about CarPlay setup is deciding how to arrange the apps on the home screen.
In roughly 5 years’ time CarPlay support has grown from 3 to 62 different auto manufacturers. I can remember shopping for my 2009 Honda (in 2012) and not seeing anything mentioned about hands-free options. Nowadays, support for CarPlay is a feature item in a lot of car sales advertisements. With more and more states enacting distracted driving legislation, I believe using these hands-free systems will eventually become mandatory.
Before we get started, let’s take a look at CarPlay’s history.
Looking in the Rearview Mirror
The concept of using an iOS device in a car goes back further than most people realize. In 2010 BMW announced support for iPod Out, which allowed a driver to use their iPod via an infotainment console in select BMW & Mini models.
The iPod connected to the car via the 30-pin to USB cable, and it would project a UI to the screen in the car. iPod Out was baked in to iOS 4, so the iPhone 3G, 3GS, 4, and the 2nd and 3rd generation iPod Touches all supported it. While BMW was the only manufacturer to support iPod Out, any auto manufacturer could have supported it; however, it just wasn’t widely advertised or adopted.
In 2012 Siri Eyes Free was announced at WWDC as part of iOS 6. Siri Eyes Free would allow a user to summon Siri (then a year old in iOS) via buttons on a steering wheel and issue any command that one could normally issue to Siri. This differed from iPod Out in that there was no need for a wired-connection. The car and iOS device (probably a phone at this point) utilized Bluetooth to communicate. The upside to Siri Eyes Free, beyond the obvious safety feature, was that it could work with any in-car system that could utilize the correct version of the Bluetooth Hands-Free Profile (HFP). No infotainment center/screen was necessary since it did not need to project a UI. A handful of auto manufacturers signed on, but widespread uptake was still absent.
At the 2013 WWDC Siri Eyes Free morphed in to iOS in the Car, which was part of iOS 7. iOS in the Car can be thought of as the parent of CarPlay, and closely resembles what we have today. There were, however, some aesthetic differences, which can be seen below.
iOS in the Car needed a wired connection to the vehicle, or so was the general thought at the time. During the iOS 7 beta, switches were found indicating that iOS in the Car could, potentially, operate over a wireless connection, and there was even mention of it possibly leveraging AirPlay (more on that later in this post). Unfortunately, iOS in the Car was not present when iOS 7 was initially released.
The following spring Apple presented CarPlay, and it was later released in iOS 7.1. At launch there were three auto manufactures that supported it: Ferrari, Mercedes-Benz, and Volvo. Personally, I cannot afford cars from any of those companies, so I am glad more manufacturers have added support.
CarPlay has changed very little since its release. iOS 9 brought wireless pairing capabilities to car models that could support it, iOS 10.3 added recently used apps to the upper left part of the screen, and iOS 12 opened up CarPlay to third party navigation applications (e.g. Google Maps and Waze). Otherwise, CarPlay’s functionality has stayed the same.
With the history lesson now over, there are a couple of things to mention. First, this research was conducted using my personal phone, an iPhone XS (model A1920) running iOS 12.2 (build 16E227). So, while I do have data sets, I will not be posting them online as I did with the Android Auto data. If you are interested in the test data, contact me through the blog site and we’ll talk.
Second, at least one of the files discussed (the cache file in the locationd path) is in a protected area of iPhone, so there are two ways you can get to it: jailbreaking iPhone or using a “key” with a color intermediate between black and white. The Springboard and audio data should be present in an iTunes backup or in an extraction from your favorite mobile forensic tool.
Let’s have a look around.
Test Drive
I have been using CarPlay for the past two and a half years. A majority of that time was with an after-market radio from Pioneer (installed in a 2009 Honda), and the last six months have been with a factory-installed display unit in a 2019 Nissan. One thing I discovered is that there are some slight aesthetic differences in how each auto manufacturer/after-market radio manufacturer visually implements CarPlay, so your visual mileage may vary. However, the functionality is the same across the board. CarPlay works just like iPhone.
Figure 8 shows the home screen of CarPlay.
The home screen looks and operates just like iPhone, which was probably the idea. Apple did not want users to have a large learning curve when trying to use CarPlay. Each icon represents an app, and the apps are arranged in rows and columns. Unlike iPhone, creating folders is not an option, so it is easy to have multiple home screens. The icons are large enough to where not much fine motor skill is necessary to press one, which means you probably won’t be hunting for or pressing the wrong app icon very often.
The button in the orange box is the home button. It is persistent across the UI, and it works like the iPhone home button: press it while anywhere and you are taken back to the home screen. The area in the blue box indicates there are two home screens available, and the area in the red box shows the most recently used apps.
Most of the apps should be familiar to iPhone users, but there is one that is not seen on iPhone: the Now Playing app. This thing is not actually an app…it can be thought of more like a shortcut. Pressing it will bring up whatever app currently has control of the virtual sound interface of CoreAudio (i.e. whatever app is currently playing or last played audio if that app is suspended in iPhone’s background).
Swiping left, shows my second home screen (Figure 9). The area in the red box is the OEM app. If I were to press it, I would exit the CarPlay UI and would return to Nissan Connect (Nissan’s telematic system); however, CarPlay is still running in the background. The OEM app icon will change depending on the auto maker. So, for example, if you were driving a Honda, this icon would be different.
A user can arrange the apps any way they choose and there are two ways of doing this, both of which are like iPhone. The first way is to press and hold an app on the car display unit, and then drag it to its desired location. The second way is done from the screen seen in Figure 10.
The screen in Figure 10 can be found on iPhone by navigating to Settings > General > CarPlay and selecting the CarPlay unit (or units – you can have multiple)…mine is “NissanConnect.” Moving apps arounds is the same here as it is on the display unit (instructions are present midway down the screen). Apps that have a minus sign badge can be removed from the CarPlay home screen. When an app is removed it is relegated to the area just below the CarPlay screen; in Figure 10 that area holds the MLB AtBat app, AudioBooks (iBooks), and WhatsApp. If I wanted to add any relegated apps to the CarPlay home screen I could do so by pushing the plus sign badge. Some apps cannot be relegated: Phone, Messages, Maps, Now Playing, Music, and the OEM app. Everything else can be relegated.
One thing to note here. iOS considers the car to be a USB accessory, so CarPlay does have to abide by the USB Restricted Mode setting on iPhone (if enabled). This is regardless of whether the Allow CarPlay While Locked toggle switch is set to the on position.
The following screenshots show music playback (Figure 11), navigation (Figure 12), and podcast playback (Figure 13).
Messages in CarPlay is a stripped-down version of Messages on iPhone. The app will display a list of conversations (see Figure 14), but it will not display text of the conversations (Apple obviously doesn’t want a driver reading while driving). Instead, Siri is used for both reading and dictating messages.
Phone is seen in Figures 15; specifically, the Favorites tab. The tabs at the top of the screens mirror those that are seen on the bottom in the Phone app on iPhone (Favorites, Recents, Contacts, Keypad, and Voicemail). Those tabs look just like those seen in iPhone.
If I receive a phone call, I can answer it in two ways: pressing the green accept button (seen in Figure 17) or pushing the telephone button on my steering wheel. Answering the call changes the screen to the one seen in Figure 18. Some of the items in Figure 18 look similar to those seen in iOS in the Car (Figure 4).
Most apps will appear like those pictured above, although, there may be some slight visual/functional differences depending on the app’s purpose, and, again, there may be some further visual differences depending on what car or after-market radio you are using.
Speaking of purpose, CarPlay is designed to do three things: voice communication, audio playback, and navigation. These things can be done fairly well through CarPlay, and done safely, which, I believe, is the main purpose. Obviously, some popular apps, such as Twitter or Facebook, don’t work well in a car, so I don’t expect true social media apps to be in CarPlay any time soon if at all (I could be wrong).
Now that we have had a tour, let’s take a look under the hood and see what artifacts, if any, can be found.
Under the Hood
After snooping around in iOS for a bit I came to a realization that CarPlay is forensically similar to Android Auto: it merely projects the apps that can work with it on to the car’s display unit, so the individual apps contain a majority of the user-generated data. Also, like Android Auto, CarPlay does leave behind some artifacts that may be valuable to forensic examiners/investigators, and, just like any other artifacts an examiner may find, these can be used in conjunction with other data sources to get a wholistic picture of a device.
One of the first artifacts that I found is the cache.plist file under locationd. It can be found in the private > var > root > Library > Caches > locationd path. cache.plist contains the times of last connect and last disconnect. I did not expect to find connection times in the cache file of the location daemon, so this was a pleasant surprise. See Figure 19.
There are actually three timestamps here, two of which I have identified. The timestamp in the red box is the last time I connected to my car. It is stored in CF Absolute Time (aka Mac Absolute Time), which is the number of seconds since January 1, 2001 00:00:00 UTC. The time, 576763615.86389804, converts to April 12, 2019 at 8:06:56 AM (EDT). I had stopped at my favorite coffee shop on the way to work and when I hopped back in the car, I plugged in my iPhone and CarPlay initialized. See Figure 20.
The time stamp in the green box just under the string CarKit NissanConnect, is a bit deceptive. It is the time I disconnected from my car. Decoding it converts it to April 12, 2019 at 8:26:18 AM (EDT). Here, I disconnected from my car, walked into work, and badged in at 8:27:14 AM (EDT). See Figure 21.
The time in the middle, 576764725.40157998, is just under a minute before the timestamp in the green box. Based on my notes, it is the time I stopped playback on a podcast that I was listening to at the time I parked. I also checked KnowledgeC.db (via DB Browser for SQLite) and found an entry in it for “Cached Locations,” with the GPS coordinates being where I parked in my employer’s parking lot. Whether the middle timestamp represents the time the last action was taken in CarPlay is a good question and requires more testing.
The next file of interest here is the com.apple.carplay.plist file. It can be found by navigating to the private > var > mobile > Library > Preferences path. See Figure 22.
The area in the red box is of interest. Here the name of the car that was paired is seen (NissanConnect) along with a GUID. The fact that the term “pairings” (plural) is there along with a GUID leads me to believe that multiple cars can be paired with the same iPhone, but I wasn’t able to test this as I am the only person I know that has a CarPlay capable car. Remember the GUID because it is seen again in discussing the next artifact. For now, see Figure 23.
Figure 23 shows the settings page just above the one seen in Figure 10. I show this merely to show that my car is labeled “NissanConnect.”
The next file is 10310139-130B-44F2-A862-7095C7AAE059-CarDisplayIconState.plist. It can be found in the private > var > mobile > Library > Springboard path. The first part of the file name should look familiar…it is the GUID seen in the com.apple.carplay.plist file. This file describes the layout of the home screen (or screens if you have more than one). I found other files in the same path with the CarDisplayIconState string in their file names, but with different GUIDs, which causes me to further speculate that multiple CarPlay units can be synced with one iPhone. See Figure 24.
The area in the red and blue boxes represent my home screens. The top-level Item in the red box, Item 0, represents my first home screen, and the sub-item numbers represent the location of each icon on the first home screen. See Figure 25 for the translation.
The area in the blue box in Figure 24 represents my second home screen, and, again, the sub-item numbers represent the location of each icon on the screen. See Figure 26 for the translation.
The entry below the blue box in Figure 24 is labeled “metadata.” Figure 27 shows it in an expanded format.
The areas in the green and purple boxes indicate that the OEM app icon is displayed, and that it is “Nissan” (seen in Figure 26). The areas in the orange and blue boxes describe how the app icon layout should be (four columns and two rows). The area in the red box is labeled “hiddenIcons,” and refers to the relegated apps previously seen in Figure 10. As it turns out, the items numbers also describe their position. See Figure 28.
Notice that this file did not describe the location of the most recently used apps in CarPlay (the area in the upper left portion of the display screen). That information is described in com.apple.springboard, which is found in the same path. See Figure 29.
Just like the app icon layout previously discussed, the item numbers for each most recently used app translate to positions on the display screen. See Figure 30 for the translation.
The next file is the com.apple.celestial.plist, which is found in the private > var > mobile > Library > Preferences path. This file had a bunch of data in it, but there are three values in this file that are relevant to CarPlay. See Figure 31.
The string in the green box represents what app had last played audio within CarPlay prior to iPhone being disconnected from the car. The area in blue box is self-explanatory (I had stopped my podcast when I parked my car). The item in the red box is interesting. I had been playing a podcast when I parked the car and had stopped playback. Before I disconnected my iPhone, I brought the Music app to the foreground, but did not have it play any music, thus it never took control of the virtual sound interface in CoreAudio. By doing this, the string in the red box was generated. Just to confirm this, I tested this scenario a second time, but did not bring the Music app to the foreground; the value nowPlayingAppDisplayIDUponCarPlayDisconnect was not present in the second plist file. I am sure this key has some operational value, although I am not sure what that value is. If anyone has any idea, please let me know.
As I mentioned earlier in this post, Siri does a lot of the heavy lifting in CarPlay because Apple doesn’t want you messing with your phone while you’re driving. So, I decided to look for anything Siri-related, and I did find one thing…although I will say that this is probably not exclusive to CarPlay. I think this may be present regardless of whether it occurs in CarPlay or not (more testing). In the path private > var > mobile > Library > Assistant there is a plist file named PreviousConversation (there is no file extension but the file header indicates it is a bplist). Let me provide some context.
When I pick up my child from daycare in the afternoons, I will ask Siri to send a message, via CarPlay, to my spouse indicating that my child and I are on the way home, and she usually acknowledges. The afternoon before I extracted the data from my iPhone (04/11/2019), I had done just that, and, after a delay, my spouse had replied “Ok.”
PreviousConversation contains the last conversation I had with Siri during this session. When I received the message, I hit the notification I received at the top of the CarPlay screen, which triggered Siri. The session went as so:
Siri: “[Spouse’s name] said Ok. Would you like to reply?”
Me: “No.”
Siri: “Ok.”
See Figure 32.
The area in the red box is the name of the sender, in this case, my spouse’s (redacted) name. The orange box was spoken by Siri, and the blue box is the actual iMessage I received from my spouse. The purple box is what was read to me, minus the actual iMessage. Siri’s inquiry (about my desire to reply) is seen in Figure 33.
Figure 34 contains the values of the message sender (my spouse). Inside of the red box the field “data” contains the iMessage identifier…in this case, my spouse’s phone number. The field “displayText” is my spouse’s name (presumably pulled from my Contact’s list). Figure 35 has the message recipient information: me.
Figure 35 also has the timestamp of when the message was received (orange box), along with my spouse’s chat identifier (blue box).
Figure 36 shows Siri’s last response to me before the session ended.
Interesting note: this plist file had other interesting data in it. One thing that I noticed is that each possible response to the inquiry “Would you like to reply?” had an entry in here: “Call” (the message sender), “Yes” (I’d like to reply), and “No” (I would not like to reply). It might be a good research project for someone. 🙂
The next artifact actually comes from a file previously discussed: com.apple.celestial.plist. While examining this file I found something interesting that bears mentioning in this post. My iPhone has never been paired via Bluetooth with my 2019 Nissan. When I purchased the car, I immediately started using CarPlay, so there has been no need to use Bluetooth (other than testing Android Auto). Under the endointTypeInfo key I found the area seen in Figure 37.
The keys in the red box contain the Bluetooth MAC address for my car. I double-checked my Bluetooth settings on the phone and the car, and the car Bluetooth radio was turned off, but the phone’s radio was on (due to my AppleWatch). So, how does my iPhone have the Bluetooth MAC address for my car? I do have a theory, so stay with me for just a second. See Figure 38.
Figure 38 shows the home screen of my iPhone while CarPlay is running. Notice that the AirPlay/Bluetooth indicator is enabled (red box). Based on some great reverse engineering, it was found that any device that uses the AirPlay service will use its MAC address in order to identify itself (deviceid). Now, see Figure 39.
Figure 39 shows two files, both of which are in the Library > Audio > Plugins > HAL path. The file on the left is the info.plist file for the Halogen driver (the virtual audio interface) for AirPlay and the file on the right is the info.plist file for the Halogen driver for CarPlay. The plug-in identifiers for each (both starting with EEA5773D) are the same. My theory is that CarPlay may be utilizing AirPlay protocols in order to function, at least for audio. I know this is a stretch as those of us that use AirPlay know that it typically is done over a wireless connection, but I think there is a small argument to be made here. Obviously, this requires more research and testing, and it is beyond the scope of this post.
Conclusion
CarPlay is Apple’s attempt at (safely) getting into your car. It provides a singular screen experience between iPhone and the car, and it encourages safe driving. While a majority of the user-generated artifacts are kept by the individual apps that are used, there are artifacts specific to CarPlay that are left behind. The app icon layout, time last connected and disconnected, and last used app can all be found in these artifacts. There are also some ancillary artifacts that may also be useful to examiners/investigators.
It has been a long time since I really dug around in iOS, and I saw a lot of interesting things that I think would be great to research, so I may be picking on Apple again in the near future.
Does CarPlay belong to screen casting technology? What kind of data is transmitted from the phone to the car?
Does CarPlay belong to screen casting technology?
What kind of data is transmitted from the phone to the car?Does CarPlay belong to screen casting technology?