MacKuba

Kuba Suder's blog on Mac & iOS development

WWDC 16

App ClipsAppKitCloudKitExtensionsFoundationiCloudLocationLoggingMacMapsPerformancePhotosPrivacySafariSwiftSwiftUIUIKitWatchKitWWDC 12WWDC 14WWDC 15WWDC 16WWDC 18WWDC 19WWDC 20

View as index

CloudKit Best Practices

Categories: CloudKit, iCloud 0 comments Watch the video

Short CloudKit overview

Apple uses CloudKit in their applications, so you can be confident that it scales, because for Apple it scales to hundreds of millions of users

CloudKit lets you focus on building your applications and not worry about building backend services for them

It provides your users automatic authentication – if the user is logged in to iCloud on their device, they don’t need to log in separately in your app

A CloudKit container now includes 3 databases:

  • public database for data visible to everyone
  • private database for a given user’s private data
  • new this year: shared database for user data that they decided to share with others

Zones:

  • public database has 1 default zone
  • private database has a default zone and it can have one or more custom zones
  • shared database includes some number of shared zones

A record always exists in a specific zone

Building an app with a sync feature

A common use case (e.g. Notes app):

  • user creates some data/records/documents on one of their devices
  • later, they open another device and they expect to see these documents there and be able to read/edit them

The way this is implemented is that CloudKit needs to be the source of truth, and the devices should maintain a local cache of all the app data and synchronize it using CloudKit

The recommended workflow:

  1. 1. On app launch, fetch changes from the server
  2. 2. Subscribe to any future changes
  3. 3. Fetch changes when you receive a push

Subscriptions:

Subscriptions let you ask the server to notify you whenever a change happens in the specified set of data. Previously you could subscribe to a specific query to a record type or to all changes in a zone.

New in iOS 10 – CKDatabaseSubscription – lets you subscribe to all changes in the whole database (private or shared).

Types of subscription notifications:

  1. 1. Silent push:
let notificationInfo = CKNotificationInfo()

// we only set this, but none of the UI related keys
notificationInfo.shouldSendContentAvailable = true

// do this once. no need to ask the user for push notifications permission,
// since we won't show any visible notifications
application.registerForRemoteNotifications(…)
  1. 2. Visual notification:
let notificationInfo = CKNotificationInfo()

// set any of these
notificationInfo.shouldBadge = true
notificationInfo.alertBody = "alertBody"
notificationInfo.soundName = "default"

// we need to prompt the user for push notification access:
application.registerUserNotificationSettings(…)
application.registerForRemoteNotifications(…)

Remember that push notifications can be coalesced, so you may only get one out of a series. Push notifications tell you that *something* has changed, but not necessarily every single thing that has changed.

Creating a subscription:

This only needs to be done the first time you launch an app – so we set a flag when we create a subscription and the next time we skip this part.

if subscriptionIsLocallyCached { return }

let subscription = CKDatabaseSubscription(subscriptionID: "shared-changes")

let notificationInfo = CKNotificationInfo()
notificationInfo.shouldSendContentAvailable = true
subscription.notificationInfo = notificationInfo

let operation = CKModifySubscriptionsOperation(
    subscriptionsToSave: [subscription],
    subscriptionIDsToDelete: []
)

operation.modifySubscriptionsCompletionBlock = { …
    if error != nil {
        …
    } else {
        self.subscriptionIsLocallyCached = true
    }
}

operation.qualityOfService = .utility
self.sharedDB.add(operation)

Listening for pushes:

  • turn on “Remote notifications” and “Background fetch” capabilities
func application(_ application: UIApplication,
    didReceiveRemoteNotification userInfo: [NSObject: AnyObject],
    fetchCompletionHandler completionHandler: (UIBackgroundFetchResult) -> Void) {

    let dict = userInfo as! [String: NSObject]
    let notification = CKNotification(fromRemoteNotificationDictionary: dict)

    if notification.subscriptionID == "shared-changes" {
        fetchSharedChanges {
              completionHandler(.newData)
        }
    }
}

Fetching the changes:

Steps:

  • ask in which zones something was changed (in shared db – because there may be new zones added when a new user shares some content)
  • ask which records have changed in each relevant zone

The server will not send you pushes about the changes you’re doing on this device, but you may receive those changes you’ve done on the list when fetching a delta download

fetchAllChanges: previously, in some operations you had to manually check for a flag that says there are more results waiting for you that you need to manually request (i.e. another page)

Now, CloudKit does the paging automatically for you if fetchAllChanges = true (which is the default)

func fetchSharedChanges(_ callback: () -> Void) {
    let changesOperation = CKFetchDatabaseChangesOperation(
        previousServerChangeToken: sharedDBChangeToken  // cached between runs
    )

    // this gives you IDs of changed zones
    changesOperation.recordZoneWithIDChangedBlock = { … }

    // this gives you IDs of deleted zones
    changesOperation.recordZoneWithIDWasDeletedBlock = { … }

    // this gives you the current change token which you need to save
    // may be called multiple times if the operation fetches multiple pages of content
    // save the token each time, so in case of an error you don"t repeat all work
    changesOperation.changeTokenUpdatedBlock = { … }

    changesOperation.fetchDatabaseChangesCompletionBlock = {
        (newToken: CKServerChangeToken?, more: Bool, error: NSError?) -> Void in

        self.sharedDBChangeToken = newToken
        self.fetchZoneChanges(callback)
    }

    self.sharedDB.add(operation)
}

fetchZoneChanges looks very similar, but fetches changes for a specific zone using CKFetchRecordZoneChangesOperation (you pass it a list of zones)

CloudKit best practices:

Automatic authentication:

CloudKit allows you to authenticate users (if they’re logged in to iCloud) without requiring any private information

You use the CloudKit user record for authentication

The user record is unique per container and never changes for that user

container.fetchUserRecordID(completionHandler: (CKRecordID?, NSError?) -> Void)

CKOperation API:

The convenience API works on single items and it’s simpler to use

Every convenience API call has a CKOperation counterpart that lets you perform an operation on a batch of records

The CKOperation also has other advantages – for example, it lets you:

  • set up dependencies between operations
  • specify quality of service and queue priorities
  • cancel operations that have started executing
  • specify if you want the operation to work over cellular network
  • limit the number of records or set of fetched keys
  • report progress
  • … and everything that NSOperation provides

(*) watch the Advanced NSOperations talk from 2015 to learn more about NSOperation

Quality of service:

QoS: select a quality of service (.userInteractive / .userInitiated / .utility / .background) depending on the task priority

  • default is .utility
  • .utility and below enable discretionary networking

Discretionary networking means that:

  • the system decides when is the best moment to run your request, so it may take longer than you expect
  • however, all network failures will be automatically retried for you
  • the request gets a timeout period of 7 days by default

Long lived operations:

If you have some operations that you want to continue/retry if they don’t manage to complete by the time your app is terminated, iOS 9.3 adds “CloudKit long lived operations”

Once you run such operation, the system will finish it even if the app is killed by the system or the user

The request is executed even if your app isn’t running, the result is cached and is returned to you once the app restarts

Results are kept by the OS for at least 24 hours

To use this API:

  • set isLongLived = true on CKOperation
  • save the operation’s operationID
  • use CKContainer.fetchLongLivedOperation(withId:) to get the operation object back
  • set completion blocks and run it again just like a new one
CKContainer.default().fetchLongLivedOperation(withID: myOpID) {
    (operation: CKOperation?, error: NSError?) in

    let fetchRecords = operation as! CKFetchRecordsOperation
    fetchRecords.fetchRecordsCompletionBlock = { … }

    CKContainer.default().privateCloudDatabase.add(fetchRecords)
}

Parent references:

A new type of reference added this year to help you better model data, especially with sharing in mind

If your app supports sharing, it’s recommended that you set the parent reference to create a hierarchy between records

Example: Album  ⭢  list of photos

let photoRecord = CKRecord(recordType: "photo")
photoRecord.setParent(albumRecordID)

What this gives you: when the user shares the album record, the whole record hierarchy under this album (photos and other data) will also be shared

Types of errors:

1) Fatal error (bad request)

Error codes like:

  • .internalError
  • .serverRejectedRequest
  • .invalidArguments
  • .permissionFailure

→ in this case, you should show an alert to the user and tell them this can’t be executed

2) Connection/server error

Error codes like:

  • .zoneBusy
  • .serviceUnavailable
  • .requestRateLimited

→ in this case, check for CKErrorRetryAfterKey and retry after specified time

3) Errors that are returned before connection is even made

.networkUnavailable

  • you should monitor network reachability (SCNetworkReachability) and retry when the device is connected again

.notAuthenticated

  • when the user is not logged in and can’t access their private database
  • you should register at startup for CKAccountChangedNotification, and when it fires, recheck account status and update the UI

Unified Logging and Activity Tracing

Categories: Foundation, Logging 0 comments Watch the video

Design of the API:

Creating one common efficient logging mechanism that can be used in user and kernel mode

Maximize information collected while minimizing the “observer effect” (affecting the program and the analyzed issue by adding logging)

Minimal footprint during the call – a lot of processing is deferred to some later time, e.g. to the moment when the information is displayed when viewing logs

Managing message lifecycle – different kinds of messages are kept around for a different duration depending on how important they are

We want as much logging happening as possible all the time without it affecting the performance of the system

Designed for privacy from the ground up

New ways of categorising and filtering log messages, so it’s easier to find messages you care about

Caller information is collected automatically, so no need to add info about the file/line etc. to the message

Built-in type specifiers that help with message formatting

New console app and command line tool

Supported across all platforms and simulators

Legacy APIs (NSLog, asl_log_message, syslog) are all redirected to this new system (if you build on the latest SDK)

All log data will be in a new location and a new format

Log data is kept in a compressed binary format, as .tracev3 files

Stored in /var/db/diagnostics and /var/db/uuidtext

You must use new tools to search the logs, since the data is binary and not searchable using grep etc.

New .logarchive format for sharing logs exported from the system log

Subsystems and categories:

Log messages can now be associated with a subsystem (e.g. an app, target or module) and a category (e.g. a section of an app), which can be used to control how messages are filtered and displayed

A subsystem can have multiple categories, and you can use as many subsystems and categories as you need

Each log message has a level:

  • 3 basic levels: Default, Info, Debug
  • 2 special levels: Fault, Error

You can set per category, per subsystem or globally logs of which level up are enabled (Default and up are always enabled) and which are stored to disk or to memory (memory log keeps a temporary space for logs that are overwritten much faster than disk logs)

Behavior can be customized globally using profiles or the log command on macOS

Default configuration is:

  • Logs of level Default and above are saved to disk
  • Info logs are enabled and stored in memory
  • Debug logs are disabled

Privacy:

The new logging system is designed to prevent accidentally logging sensitive personal information to the logs

Dynamic data injected into log messages is assumed to be private, static strings are not private

Errors and faults:

The system collects some additional information when logging errors and faults to help with investigating issues

Error  ⭢  an issue discovered within the given application/library

On an error, all previous logs from this process logged to memory are saved to disk so that they can be kept for longer

Fault  ⭢  a more global problem in the system

On a fault, all logs from this process and other processes involved in the same activity are saved to disk, plus some other system information is collected

Faults and errors and the information collected with them are saved to separate log files, so the normal logs don’t push them out

How the system works:

Within each process there are some buffers for logging messages

There is a logging daemon logd in the system, when the buffers fill up it compresses them into a larger buffer that it maintains

If you request a live stream of logs, the logs are sent to diagnosticd daemon which sends them to the client

There is a large performance hit when doing live streaming of logs, since all the optimizations can’t be used

The Console app:

Sidebar shows different kinds of log sources and connected devices

“Activities” button lets you turn on activities mode that shows a path of logs connected to a specific activity

When streaming live, by default Debug and Info messages are not printed – enable them in the menu to see them

Colored dots on the left side show message level (no dot = default level)

Use left/right arrows to expand and collapse messages in the list

To filter messages, you can search for a string in any field, or you can ask to exclude rows from a given process/subsystem/category etc.

Use the context menu to show/hide messages of the same category/process etc. as the highlighted one

Type e.g. “proc:myapp” to quickly search for process = myapp

You can save frequently used searches using the “Save” button, they’re added to the filter bar above the table

Activities can also be filtered in a similar way (however, they don’t have subsystems, categories or levels)

New APIs for logging:

os_log – writes a log of default level

os_log_info, os_log_debug, os_log_error, os_log_fault – different levels

os_log_create – creates a log object that you can customize

os_log_t log = os_log_create("com.yourcompany.subsystem", "network");

Then you pass this log object to the calls listed above:

os_log(log, "Something happened")

You can have multiple log objects for different categories and use the right one for each log

If you don’t want to have a custom subsystem/category, pass OS_LOG_DEFAULT as the log object

Log API includes built-in formatters for various types

Converting a value into a string representation is deferred to when the message is displayed in the Console

Timestamps: %{time_t}d

Error codes: %{errno}d

Arbitrary binary data: %.*P

etc.

Privacy:

Privacy is handled on a per parameter basis

Static strings are assumed to be public, dynamic strings and objects are assumed to be private unless you override it

Overriding an object to be public: %{public}@

Overriding a simple value to be private: %{private}d

You can combine privacy and formatting: %{public, uuid_t}.16P

Activities:

Activities are now objects that you can store and reuse

You can easily control the relationships between activities

os_activity_create – creates an activity object (you can pass it a parent activity to create a hierarchy)

os_activity_label_useraction – marks the activity as a “user action” (initiated by the user)

os_activity_apply(activity, ^{ … }) – executes that block as a part of that activity

os_activity_scope(activity) – executes everything until the closing brace as a part of that activity

The log command line tool:

Same functionality as the Console, but from the command line

log stream – live streaming logs

log stream --predicate 'eventMessage contains "my message"'

log show system_logs.logarchive – shows logs from a file

log config --mode "level:debug" --subsystem com.mycorp.app – enable debug logging for the given subsystem

Logging tips:

Don’t add any extra whitespace or formatting to logs

Let the SDK do string formatting

Avoid wrapping os_log calls in some custom APIs (or use macros, not functions)

Only log what’s needed from arrays and dictionaries, they can take a lot of space on disk

Avoid logging in tight code loops

How to use various levels:

  • default – for normal logs about what the app is doing
  • info – for additional info that gets stale quickly and will only matter in case of an error
  • debug – for high volume logging during development
  • error – to capture additional information from the app
  • fault – to capture additional information from the system

Collecting logs for bug reports:

The preferred method is sysdiagnose

Sysdiagnose collects logs in a file named system_logs.logarchive

You can use a specific key combination on the given device to trigger a sysdiagnose, and then transfer using iTunes

Deprecations:

All ASL logging APIs are deprecated

Older activity APIs: os_activity_start, os_activity_end, os_activity_set_breadcrumb, os_trace_with_payload