From: http://www.raywenderlich.com/17927/how-to-synchronize-core-data-with-a-web-service-part-2
This is a post by iOS Tutorial Team Member Chris Wagner, an enthusiast in software engineering always trying to stay ahead of the curve.
Welcome back to our 2-part tutorial series on how to synchronize core data with a web service!
Just to refresh your memory, here’s what you did in the first part of this series:
- Downloaded and ran the starter project
- Setup a free Parse account
- Wrote an AFNetworking client to talk to the Parse REST API
- Created a “Sync Engine” Singleton class to handle synchronization
- Processed web service data into Core Data
- Manually triggered sync with remote service
The net result of all that hard work above was that you ended up with an App that tracks important dates, and synchronizes that data with the online storage service. While that’s incredibly cool, you can make that App even more awesome by completing this second and final part of the series!
Here you will complete three more vital pieces to round out your App:
- Delete local objects when deleted on server
- Push records created locally to remote service
- Delete records on server when deleted locally
If you did not complete part 1, lost your project, or just want to start the tutorial knowing your code is in sync, don’t sweat it! :] You can download everything covered in Part 1 here.
If you do choose to use this file, make sure to replace values for kSDFParseAPIApplicationId and kSDFParseAPIKey with the values provided to you from the Overview tab of the Parse project window. Also, make sure to build and run the program before going any further just to make sure that everything is in working order.
Ready? Let’s dive in to deletion!
Delete local objects when deleted on server
To be sure that local objects are deleted when they no longer exist on the server, your app will download all of the records on the server and compare them with what you have locally. It’s assumed that any record you have locally, that does not exist on the server, should be deleted.
One untoward side effect with the Parse REST API is that this approach causes some overhead as it retrieves the full objects, instead of just the objectID fields. (Holy data usage, Batman!) Alternatively, you could have a delete flag on your records on the remote service and retrieve all records matching with the deleted flag set. While this approach would reduce overhead, the downside is that you can never actually delete records from the server — the records will stick around in perpetuity.
First, in SDSyncEngine.m, update the signature of:
- (void)downloadDataForRegisteredObjects:(BOOL)useUpdatedAtDate { |
To be:
- (void)downloadDataForRegisteredObjects:(BOOL)useUpdatedAtDate toDeleteLocalRecords:(BOOL)toDelete { |
And then update your -startSync method to reflect the new signature:
- (void)startSync { if (!self.syncInProgress) { [self willChangeValueForKey:@"syncInProgress"]; _syncInProgress = YES; [self didChangeValueForKey:@"syncInProgress"]; dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{ [self downloadDataForRegisteredObjects:YES toDeleteLocalRecords:NO]; }); } } |
Now you need a method to process the deletion of these local records. Add the following method below -processJSONDataRecordsIntoCoreData:
- (void)processJSONDataRecordsForDeletion { NSManagedObjectContext *managedObjectContext = [[SDCoreDataController sharedInstance] backgroundManagedObjectContext]; // // Iterate over all registered classes to sync // for (NSString *className in self.registeredClassesToSync) { // // Retrieve the JSON response records from disk // NSArray *JSONRecords = [self JSONDataRecordsForClass:className sortedByKey:@"objectId"]; if ([JSONRecords count] > 0) { // // If there are any records fetch all locally stored records that are NOT in the list of downloaded records // NSArray *storedRecords = [self managedObjectsForClass:className sortedByKey:@"objectId" usingArrayOfIds:[JSONRecords valueForKey:@"objectId"] inArrayOfIds:NO]; // // Schedule the NSManagedObject for deletion and save the context // [managedObjectContext performBlockAndWait:^{ for (NSManagedObject *managedObject in storedRecords) { [managedObjectContext deleteObject:managedObject]; } NSError *error = nil; BOOL saved = [managedObjectContext save:&error]; if (!saved) { NSLog(@"Unable to save context after deleting records for class %@ because %@", className, error); } }]; } // // Delete all JSON Record response files to clean up after yourself // [self deleteJSONDataRecordsForClassWithName:className]; } // // Execute the sync completion operations as this is now the final step of the sync process // [self executeSyncCompletedOperations]; } |
Next, update the implementation of -downloadDataForRegisteredObjects:toDeleteLocalRecords: to take into consideration this new BOOL, toDelete.:
...
[[SDAFParseAPIClient sharedClient] enqueueBatchOfHTTPRequestOperations:operations progressBlock:^(NSUInteger numberOfCompletedOperations, NSUInteger totalNumberOfOperations) {
} completionBlock:^(NSArray *operations) {
if (!toDelete) {
[self processJSONDataRecordsIntoCoreData];
} else {
[self processJSONDataRecordsForDeletion];
}
}];
...
|
The only changes are in completionBlock for enqueueBatchOfHTTPRequestOperations..
Now that -processJSONDataRecordsIntoCoreData is no longer the last method to be executed in the sync process, you must remove the following line from its implementation:
[self executeSyncCompletedOperations]; |
The final lines of this method should now look like:
...
[managedObjectContext performBlockAndWait:^{
NSError *error = nil;
if (![managedObjectContext save:&error]) {
NSLog(@"Unable to save context for class %@", className);
}
}];
[self deleteJSONDataRecordsForClassWithName:className];
}
[self downloadDataForRegisteredObjects:NO toDeleteLocalRecords:YES];
}
|
Now build and run the App! Once the App is running, go to the Parse Data Browser and delete one of your records. After deleting the record go back to the App and press the Refresh button to see it disappear! Holy cow — like magic, it’s gone! Now you can add and remove records from the remote service and your App will always stay in sync.
Push records created locally to remote service
In this section, you will create a feature that will push records created within the App to the remote service. Start by adding a new method to SDAFParseAPIClient to handle this communication. Open SDAFParseAPIClient.h and add the following method declaration:
- (NSMutableURLRequest *)POSTRequestForClass:(NSString *)className parameters:(NSDictionary *)parameters; |
Now implement this method:
- (NSMutableURLRequest *)POSTRequestForClass:(NSString *)className parameters:(NSDictionary *)parameters { NSMutableURLRequest *request = nil; request = [self requestWithMethod:@"POST" path:[NSString stringWithFormat:@"classes/%@", className] parameters:parameters]; return request; } |
This new method simply takes a className and an NSDictionary of parameters which is your JSON data to post to the web service. Take a look at creating objects with the Parse REST API for some background on what you’ll be doing in this next step. Essentially, it is a standard HTTP POST — so if you have some web experience with HTTP operations, this should be very familiar!
Next you need a way to determine which records are to be pushed to the remote service. You already have an existing syncStatus flag in SDSyncEngine.h which can be used for this purpose. Update your enum to look like:
typedef enum { SDObjectSynced = 0, SDObjectCreated, } SDObjectSyncStatus; |
You will then need to use this new flag when objects are created locally. Go to SDAddDateViewController.m and import the SDSyncEngine header so that the enum is visible:
#import "SDSyncEngine.h"
|
Next, update the -saveButtonTouched: method to set the syncStatus flag to SDObjectCreated when a new record is added:
- (IBAction)saveButtonTouched:(id)sender { if (![self.nameTextField.text isEqualToString:@""] && self.datePicker.date) { [self.date setValue:self.nameTextField.text forKey:@"name"]; [self.date setValue:[self dateSetToMidnightUsingDate:self.datePicker.date] forKey:@"date"]; // Set syncStatus flag to SDObjectCreated [self.date setValue:[NSNumber numberWithInt:SDObjectCreated] forKey:@"syncStatus"]; if ([self.entityName isEqualToString:@"Holiday"]) { ... |
When a new record is added, it will be handy to attempt to push the record to the remote service immediately, in order to save the user a sync step later. Update the addDateCompletionBlock in -prepareForSegue:sender in order to call startSync in the addDateCompletionBlock to immediately push the record to the remote service.
[addDateViewController setAddDateCompletionBlock:^{ [self loadRecordsFromCoreData]; [self.tableView reloadData]; [[SDSyncEngine sharedEngine] startSync]; }]; |
In order to send your Core Data records to the remote service you must translate them to the appropriate JSON format for the remote service and use your new method in SDAFParseAPIClient. The JSON string for Holidays and Birthdays will be different, so you will need two different methods. In order to keep the sync engine decoupled from your Core Data entities, add a category method on NSManagedObject which can be called from the sync engine to get a JSON representation of the record in Core Data.
Go to File\New\File…, choose iOS\Cocoa Touch\Objective-C category, and click Next. Enter NSManagedObject for Category on, name the new category JSON, click Next and Create.
You will now have two new files, NSManagedObject+JSON.h and NSManagedObject+JSON.m. Add two new method declarations in NSManagedObject+JSON.h:
- (NSDictionary *)JSONToCreateObjectOnServer; - (NSString *)dateStringForAPIUsingDate:(NSDate *)date; |
This method will return an NSDictionary which represents the JSON value required to create the object on the remote service. You will use NSDictionary as it is easy to create in pure Objective-C syntax and it is what your POST method in SDAFParseAPIClient expects. AFNetworking will take care of the task to convert the NSDictionary to a string for you when sending the POST request to the server.
Implement the category method in NSManagedObject+JSON.m:
- (NSDictionary *)JSONToCreateObjectOnServer { @throw [NSException exceptionWithName:@"JSONStringToCreateObjectOnServer Not Overridden" reason:@"Must override JSONStringToCreateObjectOnServer on NSManagedObject class" userInfo:nil]; return nil; } |
This looks like a rather odd implementation, doesn’t it? The issue here is that there is no generic implementation possible for this method. ALL of the NSManagedObject subclasses must implement this method themselves by overriding it. Whenever a NSmanagedObject subclass does NOT implement this method an exception will be thrown – just to keep you in line! :]
Note: A word of caution – in this next step, you’re about to edit some derived files. If you edit your Core Data model and regenerate these defined files your changes will be lost! It’s highly annoying and time-wasting when you forget to do this, so be careful! One way to get around this problem is to generate a category on the NSManagedObject subclass just as you did for NSManagedObject+JSON; all of your custom methods go in the category and you won’t lose them when you regenerate the file. You know what they say — a line of code in time saves nine…or something like that! :]
Open Holiday.m and import the category and sync engine headers:
#import "NSManagedObject+JSON.h" #import "SDSyncEngine.h" |
Now go ahead and Implement the -JSONToCreateObjectOnServer method:
- (NSDictionary *)JSONToCreateObjectOnServer { NSDictionary *date = [NSDictionary dictionaryWithObjectsAndKeys: @"Date", @"__type", [[SDSyncEngine sharedEngine] dateStringForAPIUsingDate:self.date], @"iso" , nil]; NSDictionary *jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys: self.name, @"name", self.details, @"details", self.wikipediaLink, @"wikipediaLink", date, @"date", nil]; return jsonDictionary; } |
This implementation is fairly straightforward. You’ve built an NSDictionary that represents the JSON structure required by the remote services API. First the code builds the required structure for the Date field, and then builds the rest of the structure and passes in your date NSDictionary.
Now do the same for Birthday.m:
#import "NSManagedObject+JSON.h" #import "SDSyncEngine.h" |
Don’t neglect to Import the category and sync engine headers:
- (NSDictionary *)JSONToCreateObjectOnServer { NSDictionary *date = [NSDictionary dictionaryWithObjectsAndKeys: @"Date", @"__type", [[SDSyncEngine sharedEngine] dateStringForAPIUsingDate:self.date], @"iso" , nil]; NSDictionary *jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys: self.name, @"name", self.giftIdeas, @"giftIdeas", self.facebook, @"facebook", date, @"date", nil]; return jsonDictionary; } |
Eagle-eyed readers will note that it’s exactly the same code, with the appropriate properties for Birthday objects instead of Holiday objects.
As noted in part 1, the Parse date format is just a teeny bit different than NSDate — but just enough to cause you a bit of extra work. You’ll need a small function to make the necessary changes to date strings. Add a method and its interface declaration to SDSyncEngine.h:
And to SDSyncEngine.m:
- (NSString *)dateStringForAPIUsingDate:(NSDate *)date { [self initializeDateFormatter]; NSString *dateString = [self.dateFormatter stringFromDate:date]; // remove Z dateString = [dateString substringWithRange:NSMakeRange(0, [dateString length]-1)]; // add milliseconds and put Z back on dateString = [dateString stringByAppendingFormat:@".000Z"]; return dateString; } |
Now to use the new category in SDSyncEngine.m:
#import "NSManagedObject+JSON.h"
|
Import your NSManagedObject JSON Category and add the following method, beneath -newManagedObjectWithClassName:forRecord:
- (void)postLocalObjectsToServer { NSMutableArray *operations = [NSMutableArray array]; // // Iterate over all register classes to sync // for (NSString *className in self.registeredClassesToSync) { // // Fetch all objects from Core Data whose syncStatus is equal to SDObjectCreated // NSArray *objectsToCreate = [self managedObjectsForClass:className withSyncStatus:SDObjectCreated]; // // Iterate over all fetched objects who syncStatus is equal to SDObjectCreated // for (NSManagedObject *objectToCreate in objectsToCreate) { // // Get the JSON representation of the NSManagedObject // NSDictionary *jsonString = [objectToCreate JSONToCreateObjectOnServer]; // // Create a request using your POST method with the JSON representation of the NSManagedObject // NSMutableURLRequest *request = [[SDAFParseAPIClient sharedClient] POSTRequestForClass:className parameters:jsonString]; AFHTTPRequestOperation *operation = [[SDAFParseAPIClient sharedClient] HTTPRequestOperationWithRequest:request success:^(AFHTTPRequestOperation *operation, id responseObject) { // // Set the completion block for the operation to update the NSManagedObject with the createdDate from the // remote service and objectId, then set the syncStatus to SDObjectSynced so that the sync engine does not // attempt to create it again // NSLog(@"Success creation: %@", responseObject); NSDictionary *responseDictionary = responseObject; NSDate *createdDate = [self dateUsingStringFromAPI:[responseDictionary valueForKey:@"createdAt"]]; [objectToCreate setValue:createdDate forKey:@"createdAt"]; [objectToCreate setValue:[responseDictionary valueForKey:@"objectId"] forKey:@"objectId"]; [objectToCreate setValue:[NSNumber numberWithInt:SDObjectSynced] forKey:@"syncStatus"]; } failure:^(AFHTTPRequestOperation *operation, NSError *error) { // // Log an error if there was one, proper error handling should be done if necessary, in this case it may not // be required to do anything as the object will attempt to sync again next time. There could be a possibility // that the data was malformed, fields were missing, extra fields were present etc... so it is a good idea to // determine the best error handling approach for your production applications. // NSLog(@"Failed creation: %@", error); }]; // // Add all operations to the operations NSArray // [operations addObject:operation]; } } // // Pass off operations array to the sharedClient so that they are all executed // [[SDAFParseAPIClient sharedClient] enqueueBatchOfHTTPRequestOperations:operations progressBlock:^(NSUInteger numberOfCompletedOperations, NSUInteger totalNumberOfOperations) { NSLog(@"Completed %d of %d create operations", numberOfCompletedOperations, totalNumberOfOperations); } completionBlock:^(NSArray *operations) { // // Set the completion block to save the backgroundContext // if ([operations count] > 0) { [[SDCoreDataController sharedInstance] saveBackgroundContext]; } // // Invoke executeSyncCompletionOperations as this is now the final step of the sync engine's flow // [self executeSyncCompletedOperations]; }]; } |
Now go to your -processJSONDataRecordsForDeletion method and replace
[self executeSyncCompletedOperations]; |
with:
[self postLocalObjectsToServer]; |
Build and run the App! Go ahead and create a new record; create BOTH a Holiday and a Birthday record if you’re feeling brave! :] After the sync finishes, go to the data browser in Parse and you should see your newly created record! It works! This sync stuff is easy; looks like it’s time to fire all the Java guys!
Delete records on server when deleted locally
Now try deleting a record (swipe to delete) and then press the refresh button.
Whoa, what’s going on here? No, unfortunately aliens are not responsible for this behaviour! The issue is that you are not tracking when an object is deleted locally and sending that information to the remote service. First you need to add another syncStatus option open SDSyncEngine.h and update your enum to reflect the following:
typedef enum { SDObjectSynced = 0, SDObjectCreated, SDObjectDeleted, } SDObjectSyncStatus; |
Next you need to add a new method SDAFParseAPIClient which will process the deletion on the remote service.
Add the following method declaration to SDAFParseAPIClient.h:
- (NSMutableURLRequest *)DELETERequestForClass:(NSString *)className forObjectWithId:(NSString *)objectId; |
Now implement the method in SDAFParseAPIClient.m:
- (NSMutableURLRequest *)DELETERequestForClass:(NSString *)className forObjectWithId:(NSString *)objectId { NSMutableURLRequest *request = nil; request = [self requestWithMethod:@"DELETE" path:[NSString stringWithFormat:@"classes/%@/%@", className, objectId] parameters:nil]; return request; } |
Next you need to flag records as deleted when the user deletes them. Open SDDateTableViewController.m and update -tableView:commitEditingStyle:forRowAtIndexPath: with the following implementation:
- (void)tableView:(UITableView *)tableView commitEditingStyle:(UITableViewCellEditingStyle)editingStyle forRowAtIndexPath:(NSIndexPath *)indexPath { if (editingStyle == UITableViewCellEditingStyleDelete) { NSManagedObject *date = [self.dates objectAtIndex:indexPath.row]; [self.managedObjectContext performBlockAndWait:^{ // 1 if ([[date valueForKey:@"objectId"] isEqualToString:@""] || [date valueForKey:@"objectId"] == nil) { [self.managedObjectContext deleteObject:date]; } else { [date setValue:[NSNumber numberWithInt:SDObjectDeleted] forKey:@"syncStatus"]; } NSError *error = nil; BOOL saved = [self.managedObjectContext save:&error]; if (!saved) { NSLog(@"Error saving main context: %@", error); } [[SDCoreDataController sharedInstance] saveMasterContext]; [self loadRecordsFromCoreData]; [self.tableView reloadData]; }]; } } |
Take a close look at the comment mark “1″. This line was removed:
[self.managedObjectContext deleteObject:date]; |
And this line was added:
if ([[date valueForKey:@"objectId"] isEqualToString:@""] || [date valueForKey:@"objectId"] == nil) { [self.managedObjectContext deleteObject:date]; } else { [date setValue:[NSNumber numberWithInt:SDObjectDeleted] forKey:@"syncStatus"]; } |
You are no longer just deleting the record from Core Data. In the new model, if the record does NOT have an objectId (meaning it does not exist on the server) the record is immediately deleted as it was before. Otherwise you set the syncStatus to SDObjectDeleted. This is so that the record is still around when it is time to send the request to the server to have it deleted.
This poses a new problem though. (Can you see it yourself, before you read on any further?)
The deleted records still appear in the list! (And no, this one isn’t due to aliens either.) This will undoubtedly confuse the user, and they will likely try to delete it over and over again. You must next update your SDDateTableViewController to not show records whose syncStatus is set to SDObjectDeleted.
Add one line in your -loadRecordsFromCoreData method:
- (void)loadRecordsFromCoreData { [self.managedObjectContext performBlockAndWait:^{ [self.managedObjectContext reset]; NSError *error = nil; NSFetchRequest *request = [[NSFetchRequest alloc] initWithEntityName:self.entityName]; [request setSortDescriptors:[NSArray arrayWithObject: [NSSortDescriptor sortDescriptorWithKey:@"date" ascending:YES]]]; // 1 [request setPredicate:[NSPredicate predicateWithFormat:@"syncStatus != %d", SDObjectDeleted]]; self.dates = [self.managedObjectContext executeFetchRequest:request error:&error]; }]; } |
The line after the comment “1″ sets a predicate on the NSFetchRequest to ignore records whose syncStatus is equal to SDObjectDeleted. (Phew! That wasn’t so hard to fix. Those aliens will have to try a little harder next time).
Now build and run the App! Attempt to delete a record; the deleted records should no longer keep reappearing in your list when you press the refresh button.
However, there’s still one problem remaining. Can you tell what you’ve neglected to do?
Take a look at Parse — the records will still exist! (Don’t even try blaming those aliens again!) You must now modify the sync engine to use your new method in SDAFParseAPIClient.
Beneath -postLocalObjectsToServer, add the following method
- (void)deleteObjectsOnServer { NSMutableArray *operations = [NSMutableArray array]; // // Iterate over all registered classes to sync // for (NSString *className in self.registeredClassesToSync) { // // Fetch all records from Core Data whose syncStatus is equal to SDObjectDeleted // NSArray *objectsToDelete = [self managedObjectsForClass:className withSyncStatus:SDObjectDeleted]; // // Iterate over all fetched records from Core Data // for (NSManagedObject *objectToDelete in objectsToDelete) { // // Create a request for each record // NSMutableURLRequest *request = [[SDAFParseAPIClient sharedClient] DELETERequestForClass:className forObjectWithId:[objectToDelete valueForKey:@"objectId"]]; AFHTTPRequestOperation *operation = [[SDAFParseAPIClient sharedClient] HTTPRequestOperationWithRequest:request success:^(AFHTTPRequestOperation *operation, id responseObject) { NSLog(@"Success deletion: %@", responseObject); // // In the operations completion block delete the NSManagedObject from Core data locally since it has been // deleted on the server // [[[SDCoreDataController sharedInstance] backgroundManagedObjectContext] deleteObject:objectToDelete]; } failure:^(AFHTTPRequestOperation *operation, NSError *error) { NSLog(@"Failed to delete: %@", error); }]; // // Add each operation to the operations array // [operations addObject:operation]; } } [[SDAFParseAPIClient sharedClient] enqueueBatchOfHTTPRequestOperations:operations progressBlock:^(NSUInteger numberOfCompletedOperations, NSUInteger totalNumberOfOperations) { } completionBlock:^(NSArray *operations) { if ([operations count] > 0) { // // Save the background context after all operations have completed // [[SDCoreDataController sharedInstance] saveBackgroundContext]; } // // Execute the sync completed operations // [self executeSyncCompletedOperations]; }]; } |
Now that this method is the last part of the sync engine’s flow you must update your -postLocalObjectsToServer method by replacing:
[self executeSyncCompletedOperations]; |
with:
[self deleteObjectsOnServer]; |
And that’s it! Congratulations – you’re done — despite that annoying alien interference! ;] Now all records created or deleted on your remote service will show up or disappear in your app. The reverse is true as well — all records created or deleted locally will be reflected on the remote service.
Where to go from here?
Here is the final example project from this tutorial series.
Even though you’ve got a pretty well-rounded app, there’s still a few ways to improve it. The next natural progression would be to add the ability for users to edit records locally; as well, any edits made on the local service should sync with the App.
There’s also the blaring omission of adding images within the App. Depending on the remote service, you could do this in a variety of ways. This is mostly an implementation detail, but is definitely something that users would want. To give you a starting point, the same POST strategy described above for pushing records to the server should be applicable.
Synchronization is an incredibly difficult task in the mobile iOS world. Depending on the amount of data that needs to be synced, the strategy outlined in this tutorial may not be optimal. Although efficiency was touched on throughout this tutorial, it’s usually best to avoid “premature optimization” – tuning your strategy to your app’s needs will always be better than cutting and pasting a canned solution.
While extending your app, if you find that memory footprints are increasing beyond acceptable levels, you should look into using autorelease pools in the processing loops. A good tip is to use Instruments to help you find the points where memory usage is high. You may also find that you should not invoke the synchronization process too often or too soon, or you may need to prevent application usage during the initial sync if you are expecting a lot of data to be coming in.
I hope that this tutorial helps you in determining the best strategy for synchronization in your apps. I also hope that I was able to make it as generic as possible so that you can use it in your real world applications.
