MacKuba

Kuba Suder's blog on Mac & iOS development

WWDC 14

App ClipsAppKitCloudKitExtensionsFoundationiCloudLocationLoggingMacMapsPerformancePhotosPrivacySafariSwiftSwiftUIUIKitWatchKitWWDC 12WWDC 14WWDC 15WWDC 16WWDC 18WWDC 19WWDC 20

View as index

Advanced CloudKit

Categories: CloudKit, iCloud 0 comments Watch the video

CloudKit API is designed to be asynchronous, all calls return through a callback, because they all require a network connection

The main API (“operational API”) is based on NSOperation

You use it by creating special NSOperation objects for a given use case, e.g. CKFetchRecordsOperation, and specifying parameters and callbacks in their properties

Apart from the final result callback, you can set callbacks e.g. for reporting download progress or to get records one by one as they’re downloaded

Operation lifecycle (cancelling, suspending etc.) can be managed through standard NSOperation methods and NSOperationQueue

There are separete fetch/modify operation types for records, subscriptions, zones, users and notifications

You can set dependencies between operations (also if they’re in different queues), e.g. make a fetch operation and then a modify operation that needs to wait for the object to load

Operations can also have different priority levels

Starting an operation:

(ℹ️ Note: this wasn’t in the video, but it really should have been, because it's completely not obvious.)

How to start an operation once you prepare the CKOperation object:

1) Use the database’s built-in queue:

let fetchOperation = ...
CKContainer.default().privateCloudDatabase.add(fetchOperation)

2) Use your own operation queue and assign a reference to the database:

let operationQueue = NSOperationQueue()

let fetchOperation = ...
fetchOperation.database = CKContainer.default().privateCloudDatabase
operationQueue.addOperation(fetchOperation)

Custom zones

Custom zones (in the private database) let you compartmentalize data and add some special features

Records can’t be moved between zones or have cross-zone relationships

There are some operations that can only be done in custom zones:

Atomic commits:

Objects in the CloudKit database have relationships between them, and you want to keep all data consistent

Atomic commits are kind of like transactions in a relational database: batch operations succeed or fail together

Only available in the private database (because public database may be accessed by millions of users at the same time)

If an operation fails, you get a CKErrorPartialFailure response, with the user info containing info about errors on specific records (CKPartialErrorsByItemID)

Error CKErrorBatchRequestFailed means that this record wasn’t saved because of a problem with another record in the batch

Delta downloads:

Allow you to download a list of all changes since the last time the app was online, to let you perform a full sync

When a device connects, you can send a “change token” to the server asking for all changes since that version

This lets you implement an offline cache of the whole dataset and sync any changes when possible

To do that:

  • track all local changes
  • send changes to the server when connected
  • resolve conflicts
  • fetch server changes with CKFetchRecordChangesOperation
  • remember the received new server change token and send it back next time

Zone subscriptions:

Let you subscribe for notifications about any change in the zone

When you get a notification, you request a delta download

Advanced record operations

Record changes:

When you change some fields in a CKRecord, the changes are automatically tracked locally and only the changed fields are transmitted when you save it

By default CloudKit performs a “locked update”, which makes sure that the update is only saved on the server if the record wasn’t modified in the meantime by another client (this uses record change tokens)

After you execute a save, the server returns your record with a new change token – so you should use that returned version for any subsequent changes

Unlocked update  ⭢  just overwrites server data regardless what is there

Locked update  ⭢  if the record was changed in the meantime, you get back an error (CKErrorServerRecordChanged)

The userInfo of the CKErrorServerRecordChanged error contains info that lets you perform a 3-way merge:

  • CKRecordChangedErrorClientRecordKey – what you tried to save
  • CKRecordChangedErrorAncestorRecordKey – the original version
  • CKRecordChangedErrorServerRecordKey – what is currently on the server

Based on the values from these 3 copies of the record you can decide what state the record should be in, and then retry the save

You can modify the behavior with “save policies”:

  • SaveIfServerUnchanged  ⭢  default, performs a locked update and sends only changed keys
  • SaveChangedKeys  ⭢  unlocked update, sends only changed keys
  • SaveAllKeys  ⭢  unlocked update, overwrites all keys in the record (note: this doesn’t affect keys that aren’t present in the local copy at all)

You should almost always use the default locked update (SaveIfServerUnchanged), use unlocked updates only to forcefully resolve serious conflicts

Use SaveAllKeys if the user requests to overwrite server data with local data

Partial records:

The desiredKeys field present in most operation types lets you specify that you only want to download selected keys from the server

This is useful if the whole record is very large and you don’t need all of it

Partial records can be normally saved after a change

CloudKit data modeling

References:

Forward reference  ⭢  a parent object keeps an array of references to children in its property

Backward reference  ⭢  only child objects have a reference to the parent

It’s recommended to use backward references – with a forward reference you need to update the parent object every time a new child is added, and you will run into conflicts if multiple clients are adding records

To get a list of all children using backward references, make a query for all child records with a predicate “owner = X”

References give you cascading deletes – when you delete the parent object, all child objects and their children are deleted

If an object has two parent references, it’s deleted when the first parent is deleted

When batch uploading a tree of objects, CloudKit makes sure that parent objects are uploaded first so that you don’t get inconsistent data during upload (important in the public database)

Your data objects:

CloudKit is only a transport mechanism and requires you to keep and manage your own local copy of all data

It’s recommended that you don’t subclass CK* objects to build your models – make your own completely independent model classes and translate to/from CloudKit objects when fetching and saving

Handling push notifications:

You need to remember that push notifications in general aren’t guaranteed to be delivered

The server only stores one push per client, so if you reconnect e.g. after a flight, you might miss some previous notifications

You can find pushes that you’ve missed in a “Notification Collection” where every notification is saved

The Notification Collection works kind of like delta updates – you ask for notifications since a given change token and you get a list of everything added since then

You can mark a notification as read, which notifies all other clients that they can ignore it

You should check the Notification Collection every time you get a push, since you never know what you might have missed (this doesn’t only happen with airplane mode)

The iCloud Dashboard

The dashboard lets you browse data saved by your app – the whole public database and the private database for your developer account (but not anyone else’s private database)

You can view saved records, run queries with any filters, and add new records

You can define roles in the public database and define for each model who can create/read/modify records (e.g. specify that records are publicly readable but only an admin can create them)

You will also see a list of all user ids and first/last names of those users that marked themselves as discoverable

Schema:

The CloudKit database has two separate “environments”: development and production

The schema for each record type is “just in time” during development, i.e. when you save a new type of record, it automatically creates a new schema for that record type, recording every field type, and when you save a record with a new field, it adds a field to the list

However, once you’re ready to release a new version of your app, you need to save the schema to production and at that point it’s locked – a production version of the app can’t save records or fields that aren’t defined in the schema

CloudKit also automatically creates indexes for each field in each record type – when you’re done with development, you can delete some indexes that you won’t need so they don’t waste space in the production database

Tips & tricks

Please handle all errors :)

Remember that you can get partial errors (when atomic commits aren’t used), so some records might be saved while others aren’t

Retry any “server busy” errors (CKErrorRetryAfterKey tells you the amount of time you should wait)

Don’t waste space in your users’ iCloud in private databases, they may be paying real money for it

Limits in the public database are mostly to prevent abuse, they should be fine for most normal use (the limits scale with the number of users)

Introducing CloudKit

Categories: CloudKit, iCloud 0 comments Watch the video

CloudKit lets you write client applications without having to build and host a server part to handle things like database, accounts or push notifications

Usage is free for the developer up to pretty big limits

CloudKit gives you more direct access to iCloud servers

It’s the framework that’s used behind the scenes by iCloud Photo Library and iCloud Drive

Uses the same iCloud account as iCloud documents or key-value storage

Two types of databases:

  • public – accessible to everyone
  • private – private data of a specific user

It’s only a transport technology, it does not deal with local data persistence – you need to decide how you store the data that you load from the cloud

To enable iCloud in your app, set it up in the Capabilities tab in Xcode just like with other iCloud APIs

Containers:

Each app’s data is kept in a separate container (CKContainer)

Containers give you the safety that your app’s data will not be mixed with someone else’s app’s data

A container’s ID needs to be unique in the whole iCloud, so use reverse-domain style identifiers

By default each app has one container of its own, but apps can additionally use shared containers

Containers are managed by the developer through the WWDR portal

Databases:

A container contains one shared public database for everyone, and separate private databases for each user

An app running on the device has access to one public and one private database (CKDatabase)

The database is the initial entry point to CloudKit (from a container)

CKDatabase *publicDatabase = [[CKContainer defaultContainer] publicCloudDatabase];
CKDatabase *privateDatabase = [[CKContainer defaultContainer] privateCloudDatabase];

Private database:

  • requires a logged in iCloud account
  • data stored counts against the user’s iCloud account quota
  • default permission for data is user readable
  • the data your users store in your app’s CloudKit container is *not* accessible to you

Public database:

  • can be accessed anonymously even if the user isn’t logged in
  • data stored counts against the developer’s app quota
  • default permission for data is world readable
  • permissions can be customized using iCloud Dashboard Roles

Records:

A record (CKRecord) is a single “object” in the CloudKit database, essentially a list of key-value pairs

Records have a Record Type (~ table name)

There is no defined up front schema, you can just save a record of any type with any keys and the schema will be updated based on that

→ note: this only works in development, the schema is fixed in production, see "Advanced CloudKit"

Records can have metadata: who created it & modified it and when, also includes a “change tag” (version id) – for determining if two sides have the same version of a record

Record values can be: strings, numbers, dates, NSData, CLLocation, CKReference, CKAsset, arrays of any of these

- (instancetype)initWithRecordType:(NSString *)recordType;
- (id)objectForKey:(NSString*)key;
- (void)setObject:(id)object forKey:(NSString *)key;
- (NSArray *)allKeys;

Subscripts also work:

CKRecord *party = [[CKRecord alloc] initWithRecordType:@"Party"];
party[@"start"] = [NSDate date];

Record zones:

Records are grouped within a database inside “zones” (CKRecordZoneID)

The public database has one zone, the private database has one default zone, but it can have additional custom zones

Record identifiers:

Record identifier (CKRecordID) is a tuple grouping: a “record name” + zone ID

You can provide a recordID when creating a record instance

If you don’t provide a recordID, a random UUID will be assigned

References:

A reference (CKReference) is a pointer from one record to another, as an id of the “parent” record contained in a child record’s field

References allow you to do cascade deletes, deleting child records when parent is deleted

You can create a reference from a CKRecord object or from a CKRecordID if you know the ID but don’t have the object in memory

Assets:

An asset (CKAsset) is an unstructured piece of data, basically a binary file

Assets are downloaded and uploaded from/to files on disk, not from memory

An asset is always owned by a record, and is deleted when the record is deleted

Transport of assets is optimized so that only the minimal amount of data is transferred

APIs:

There are two different APIs for managing CloudKit data: “operational API” and “convenience API”

The operational API has every possible operation you might need, the convenience API is more convenient

Start with the convenience API, use operational API for tweaking and overriding options if needed

CloudKit APIs for saving/fetching data are asynchronous – there is no SDK-managed local data, everything needs to go over the network unless you manually cache it

In CloudKit it’s absolutely necessary to properly handle error cases – every network call can fail and your app needs to be prepared for this

Convenience API:

[publicDatabase saveRecord:obj completionHandler: { … }];
[publicDatabase retchRecordWithID:recordID completionHandler: { … }];

Queries:

For any large database, or the shared public database, you shouldn’t try to keep a copy of the whole database on disk and sync all of it, but instead fetch what you need on demand – for this you can use queries

A query (CKQuery) allows you to fetch a list of records matching some conditions

Query can specify a RecordType, NSPredicate and optionally NSSortDescriptors

A subset of NSPredicate language is supported, if something is not supported you’ll get an exception

Predicates such as “equal”, “greater than”, “distance to location”, string tokenizing, and OR / AND are supported

[publicDatabase performQuery:query inZoneWithID:nil completionHandler: { … }];

Subscriptions:

If you repeatedly run the same query, polling for the same data, you can ask the server to run the query for you and notify you immediately when a new record is added

You do that by creating a subscription (CKSubscription)

A subscription includes: RecordType, NSPredicate and push configuration (CKNotificationInfo)

Your app is notified of changes through a push notification with some additional data

CKSubscription *subscription =
  [[CKSubscription alloc] initWithRecordType:@"Party"
                                   predicate:predicate
                                     options:CKSubscriptionOptionsFiresOnRecordCreation];

[publicDatabase saveSubscription:subscription completionHandler: { … }];

Pushes are handled through the usual push API:

- (void)application:(UIApplication *)application
    didReceiveRemoteNotification:(NSDictionary *)userInfo;

Build a CKNotification object from the user info:

CKNotification *cloudKitNotification =
    [CKNotification notificationFromRemoteNotificationDictionary:userInfo];

NSString *alertBody = cloudKitNotification.alertBody;

if (cloudKitNotification.notificationType == CKNotificationTypeQuery) {
    CKQueryNotification *queryNotification = cloudKitNotification;
    CKRecordID *recordID = [queryNotification recordID];
}

Handling user accounts:

Your application does not get direct access to any user identifiers like iCloud email address

Instead, in each container each user is represented as a unique ID within that container that doesn’t change unless the user switches to another account

The same user will have a different ID in a different CloudKit container

The ID is an instance of CKRecordID

[[CKContainer defaultContainer] fetchUserRecordIDWithCompletionHandler: { … }];

Each user has a user record representing them (which is almost like any other record) with their user id and record type = CKRecordTypeUserRecord, one in the private database, and another with the same ID in the public database

You can set and read any key-value data on this record like on other records

However, these records aren’t created by you and can’t be queried to get a list of users

[publicDatabase fetchRecordWithID:userRecordID completionHandler: { … }];

User discovery:

You can ask the user to allow you to make them discoverable by other users (they get a request popup)

If they agree, they can be looked up by user ID, specific email, or by fetching a list of all users matching your user’s contacts from the address book (this doesn’t give your app access to the address book itself, just a list of matching users)

You get back record IDs, first & last names of users, but no emails

[defaultContainer discoverAllContactUserInfosWithCompletionHandler: { … }];

Returns an array of CKDiscoveredUserInfo objects with properties userRecordID, firstName, lastName

[note: this API has been replaced since then with a new one that returns CKUserIdentity objects]

When to use CloudKit vs. other APIs?

CloudKit doesn’t replace or deprecate any existing iCloud APIs [yet ;P], it’s just an additional tool

Key-value store:

  • asynchronous, small amounts of data
  • mostly for application preferences

iCloud Drive:

  • works on files and folders
  • on OSX it makes a full offline cache of the drive
  • good for document-centric apps

iCloud Core Data:

  • built on top of iCloud Drive
  • good for keeping private, structured data (custom databases) in sync
  • note: the whole data set is downloaded to each device

CloudKit:

  • good for sharing public data between users, both structured data and large files
  • good for large data sets where not every device needs to have a copy of the whole database
  • for attaching some data to the user’s identity and sharing info between users that know each other
  • more low-level, your app is in control of when any information is downloaded or uploaded to the iCloud servers, and has responsibility for handling sync

Fix Bugs Faster Using Activity Tracing

Categories: Foundation, Logging 0 comments Watch the video

⚠️ The APIs mentioned in this talk seem to be somewhat abandoned – they were never made available to Swift, some things have been deprecated or have stopped working.


Apple has been adding a lot of tools for asynchronous development, like XPC, GCD, NSOperation

However, splitting work between different threads and especially between different processes makes it difficult to debug and diagnose issues

A single call to a system API might result in multiple system daemons calling each other to execute this single activity

Current logging mechanisms are insufficient, because they lack context showing how you got to the place where something happens

When something goes wrong in one of the systems involved, the whole activity fails, but it’s not clear where to look for the problem

Goals of this new tool:

  • reduce the time guessing what happened where, how did it get there and why
  • understand interaction of multiple actions over time
  • it should be lightweight and easy to use

Activities

Activity = a set of work triggered by a specific action, going across multiple systems cooperating to realize a specific goal

There’s a new system daemon diagnosticd that handles the activity tracing for all processes

Each activity has an identifier (AID) automatically propagated across the system

Activities are automatically created by UIKit and AppKit for UI actions

When you e.g. press a button in the UI, AppKit or UIKit automatically creates a new activity with some unique name, calls os_activity_start() on it, calls your IBAction handler, and then calls os_activity_end() at the end

You can also create activities explicitly:

os_activity_initiate("Activity Name", flags, ^{ … });
  • name must be a constant string
  • another variant available without a block
  • included in <os/activity.h>

Detached activity (OS_ACTIVITY_FLAG_DETACHED) = a new activity that’s not a subactivity of the current one, but a completely independent activity

Breadcrumbs

Breadcrumb = a user-facing or user-initiated action that triggered an activity, e.g. “Send an email”

A way to label activities meaningful to you

Adding a breadcrumb:

os_activity_set_breadcrumb("composing email");
  • name must be a constant string
  • only supported in the main process binary, not in libraries or plugins

⚠️ Replaced later with os_activity_label_useraction, only for activities started from IBActions

Trace messages:

Trace message = a single log in a given system as a part of an activity

New API to add trace messages to the current activity

Messages are stored in an in-memory ring buffer

Different behavior for production code and debugging

os_trace("Something happened at level %d", index);
  • only scalar format types allowed (ints, longs etc.)
  • strings and characters are *not* supported for privacy, security and performance reasons
  • included in <os/trace.h>
os_debug_trace("Open socket %d total time %f", sock, time);
  • version for debug logging
  • only actually logged if you’re in the debug mode, ignored in release mode
  • records more information – increased buffer size
  • you can enable debug mode at launch by setting env variable OS_ACTIVITY_MODE=debug

If you want to pass some additional data with a trace message, you can use os_trace_with_payload:

os_trace_with_payload("Interface: %ld", index, ^(xpc_object_t dict) {
    xpc_dictionary_set_string(dict, "interface_name", ifname);
});
  • blocks are only called and have their data recorded if the activity logs are being currently live streamed to a log viewing tool
  • uses XPC to deliver data to diagnosticd
  • you can enable stream mode with an env variable OS_ACTIVITY_MODE=stream

⚠️ Replaced later with os_log

Crash logs now include information about the current activity at the moment of the crash (name, running time etc.), the last few breadcrumbs, and recent trace messages from that activity (!)

⚠️ They don’t anymore :(

Logging errors into trace messages:

os_trace_error("Interface %ld failed with error %d", ifindex, errno);
  • soft errors, something that went wrong
os_trace_fault("Invalid state %d - will likely crash", state);
  • fatal errors, this shouldn’t happen, about to crash
  • sends trace messages from the process’s local buffer to diagnosticd

Limits:

  • format string cannot exceed 100 characters
  • formatted trace message string also has a limit, will truncate if exceeded
  • up to 7 parameters allowed
  • process ring buffer size depends on the platform and debug/release mode

Debugger: thread info includes current activity info and last trace messages

ostraceutil command line tool – live streaming of activity trace from a process (by name or pid)

⚠️ Replaced later with log

Tip: think about privacy when adding trace messages – never trace identifying information about a user or device