As I’m writing more Rust code these days, it is fascinating to see how having or not having a standardized feature or concept in version 1.0 of the language can fundamentally change the language’s surrounding ecosystem.

While it is easy to point to Rust’s borrow checker and the ownership and lifetime rules of the language, there are more generally understood features and concepts which could apply to most existing programming langauges.

Take the Result type.

Result

Conceptually, Result encapsulates return values from a function. Result is usually implemented as an enum with Ok/.success and Err/.failure variants. Both variants have associated values; for example, a Result<Person, IOError> could associate a Person value in the success case or an IOError value in the failure case. Functions can return a Result, and the function caller can exhaustively check the returned Result’s value and handle the success and failure scenarios.

Here are very contrived examples:

In Rust:

struct CustomError {
    message: String,
}

fn custom_sqrt(value: f64) -> Result<f64, CustomError> {
  if value < 0.0 {
    Err(CustomError { message: String::from("the value was negative.") })
  } else {
    Ok(value.sqrt())
  }
}

fn caller() {
  let x: f64 = 2.0;
  let computed_result = custom_sqrt(x);
  match computed_result {
    Ok(computed_value) => println!("Hooray. Got value: {}", computed_value),
    Err(error) => println!("Uh-oh, something went wrong: {}", error.message),
  }
}

In Swift:

struct CustomError: Error {
    var message: String
}

func customSqrt(_ value: Double) -> Result<Double, CustomError> {
  guard value >= 0.0 else {
    return .failure(CustomError(message: "The value was negative"))
  }

  return .success(sqrt(value))
}

func caller() {
  let x: Double = 2.0
  let computedResult = customSqrt(x)
  switch computedResult {
  case let .success(computed_value):
    print("Hooray. Got value: \(computed_value)")
  case let .failure(error):
    print("Uh-oh, something went wrong: \(error)")
  }
}

Result is basically implemented like:

In Rust:

pub enum Result<T, E> {
  Ok(T),
  Err(E),
}

In Swift:

public enum Result<S, F: Error> {
  case success(S),
  case failure(F)
}

Note that in both languages, Result is implemented like any other enum and does not require any special compiler support. 4 lines of code to define the essential functionality of the type.

In the end, Result is a basic type.

Result is in Rust 1.0’s standard library, and the type has had a profound impact on Rust’s ecosystem. In contrast, Result was added to Swift 5.0’s standard library, and the type now has to content with other ways of handling errors.

Rust

In Rust code, Result is one of the most common types used. It is used in Rust’s standard library and many third party libraries. It is such a common type that the ? operator was added in Rust 1.13 to make Result (and Option) easier to use. In practically all Rust code, if there is a function that can either return a successful value or an error, the function has a Result return type.

Swift

On the other hand, in Swift code, there are multiple ways defined to return success and failure results from functions:

Returning a normal value but throwing an error

func couldThrowSomething() -> Int throws {
  if Bool.random() {
    return 1
  } else {
    throw CustomError(message: "Uh-oh")
  }
}

func caller() {
  do {
    let value = try couldThrowSomething()
  } catch {
    // Magical "error" variable
    print("Error was \(error)")
  }
}

Returning a tuple with optional success and error values:

func compute() -> (Int?, Error?) {
  if Bool.random() {
    return (1, nil)
  } else {
    return (nil, CustomError(message: "Uh-oh"))
  }
}

func caller() {
  let result = couldThrowSomething()
  switch result {
  case let (.some(value), nil):
    print("Success. Got: \(value)")
  case let (nil, .some(error)):
    print("Error. Got: \(error)")
  case (nil, nil), (.some(_), .some(_)):
    fatalError("There should be either a success or a failure value and also not both")
  }
}

While less commonly used, the style of returning optional success and error values is seen in asynchronous callback APIs.

An example (which predates Swift but shows how multiple optional return values can be an issue) is in the infamous completion block of URLSession. Unless a developer carefully reads and understands the documentation, it is not trivial to understand exactly when a value will be present.

Returning a custom Result type

Several developers independently created their own Result implementations. However, only APIs under their control would return their Result type, and there is a potential of a conflict if their projects imported dependencies and each had their own Result types. Often this would require a mapping function which mapped custom Result to another custom Result.

Swift developers have embraced all 3 styles of returning values from functions. Now a Swift developer has to contend with things like:

  • Which style of error handling to use?

  • Figuring out the error style of each API called.

  • Mapping the error style of APIs to the preferred error style.

The post-1.0 addition of Result to Swift may cause some issues, but at least Swift has an Optional type built into the language from the initial version.

Optional

Java has lacked an Optional type until Java 8 ( which was released 18 years after Java 1.0). Any Java developer is certainly familiar with NullPointerExceptions. While Swift’s late adoption of Result may lead to some issues, null references have been called the billion dollar mistake (if not many times more). Optional is not a silver bullet towards solving all of the issues caused by null references, but it probably could have saved at least a few hundred million dollars.

Unfortunately, the lack of a feature or concept in 1.0 of a langauge may have an irreversible impact to the language’s ecosystem. Nevermind the herculean task of creating a Java standard library that has the Optional concept as part of its core design, there is too much existing code that would also need to be converted, so NullPointerExceptions will likely always remain part of a Java developer’s life.

Of course, lacking features in 1.0 of a language does not mean the language is doomed. Java has enabled corporations to make billions of dollars. Java has been adopted by many developers and is actively used by more developers than Swift and Rust developers combined today. Developers and users have extracted enormous value from using Java. If anything, Java proves that minimum viable products do exist for programming languages.

Still, 4 lines of code made many years ago does have an impact especially on my current day to day programming. Rust would not be the same without the Result type. While Rust and Swift are actually more alike than any of the other modern mainstream programming languages, there are seemingly small fundamental 1.0 details which have caused outsized divergences between the languages and their ecosystems.

One of the “secrets” to providing and maintaining great service integrations is identifying clear technical owners (who may be different than the business owner) from all parties. Whether it be internal or external third party services, having a means to contact the counterpart in the relationship is absolutely necessary.

The owner may or may not be a single person. It could be a team mailing list, but having a known point of contact is essential to great services.

For instance, if a service is expecting to change the API (for breaking changes or just additions), it would be good for the service provider to be able to inform the service consumer. Maybe there’s expected maintainence. Maybe there is now a (better) replacement service. Or maybe there’s different technical requirements (security changes). Maybe the service consumer is hammering the provider with an unexpected amount of traffic.

Of course, a service consumer should be able to contact the service provider for support. Maybe there’s an issue attempting to reach the service. Maybe there’s problems integrating or features that would make life easier.

Having contact information for both the service provider and consumer seems obvious, but often services do not invest in making sure the contacts are the intended audience, ensuring contact details are up-to-date, and the appropriate medium.

Separation of Business vs Technical

Many services fail to distinguish the technical role versus the business role. When you sign up for a third party service account, often you only provide a single email address or phone number. While the person owning the business relationship with the service provider may be technical, there is often someone else who is suppose to own the technical role.

Business owners may not have a clear understanding of technical details. So if the service provider were to contact the service consumer, the business owner may not understand the email and fail to take the appropriate action. For instance, a service provider may state that integrations are required to use a specific version of an SDK. Unless the business owner is technical enough to understand, the news may be lost and a last-minute change may be required by the service consumer. There should be nothing stopping the business owner from also receiving technical information; when adding a technical contact, the intent is not to skip the business owner, but to ensure that information reaches the relevant audience. In a way, it saves the business owner work from forwarding emails.

Ensuring up-to-date contact details

Services should build in an automated system to ensure that contact details are up to date for all relevant parties. Often, technical owners change if not business owners. Unfortunately, many times due to reorgs or other personnel changes, the responsibilities are shuffled and practically no one will/should volunteer to take on more responsibilities without any clear benefits.

For small businesses, a contractor may be involved to develop a new feature using a third party service, but when the contractor is finished with their work, the contractor is no longer the appropriate person to contact. Unfortunately, the service provider is not given the right contact information so any news from the service provider is subsequently lost.

When the contact information is out of date, it leads to unexpected outages, emergency patches, and other stressful events. All because one party could not contact the other.

Whether it be reminding contacts to update their information every few months, having tools to ensure transfer of ownership, or other means, service providers and consumers should be forced to regularly keep their contact information up to date with their respective counterpart.

Appropriate medium for contact

In short, email. While there are many chat programs and services, email is nearly universal for all parties. The communication should not be that often so email is appropriate in many ways.

Each service can decide what is appropriate to contact the other party with (e.g. new API capabilities, SDK versions, requirement changes, or maybe only important emergency notices).

Blogs, GitHub issues, notifications from RSS feeds for version releases, and other mediums are nice, but in many situations neither party can count on them as being monitored by the other party. Email may also be ignored but it is practically the only universal medium where direct contact can be made.

In conclusion, hopefully this post has given some ideas about how to effectively communicate service providers and consumers and why having that relationship be maintained helps ensure a smooth service integration.

Synchronization has been on my mind recently. I have to build a sync engine, primarily on Apple platforms. While the ultimate goal is to make sure that a user can access and modify data on any device, there are many implementation details which have different tradeoffs.

For instance, how do various different devices communicate changes between each other? The general assumption is a network connection will transmit some form of state. Is the state transmitted via HTTP or some other protocol? Is the complete state of the device transmitted or only changes made on the device? Should data be synchronized in real-time or can the data be synchronized with a delay?

Beyond how the state is transmitted, is each device a complete view of the entire data set? Does each device even have the capacity for all of the data? If each device is only a partial view of the data, is there a single source of truth? How does a device determine what is relevant data if it only has a partial view, and how does it obtain the relevant data?

Do devices have constant and reliable network connectivity? If they do not, how does data get merged together when a device cannot communicate? While the device is unreachable, it may make changes and other devices could make changes as well. If there are conflicting changes made, how do the conflicts get resolved?

These questions and decisions are only a few of the considerations which will have major effects on how the synchronization is made.

To clarify which choices to make, there are other desired goals when synchronizing data:

  • The data transmitted to and from a device should only be what is absolutely necessary. Network resources can be fairly scarce especially on mobile devices. Whether on Wi-Fi, on cellular networks, or on wired ethernet, using the network should be considered relatively expensive. Furthermore, any data sent or received requries power to process, so it is better to only process what is necessary.

  • Synchronization data should be batchable. With network resources being considered scarce, the number of network requests should be limited. Therefore, any data changes should be able to be batched together as one network request (whether data is transmitted to or from a device).

  • Devices should be able to receive a partial set of the synchronization data and make progress towards the latest state. If a device received only half of the data updates required to transition the device to the latest state, it should still be able to process the updates received and transition the local device state. The synchronization updates could be large enough either in quantity or size that a device may need to process multiple batches.

  • The processing of synchronization data should be streamable. Whether there is one other device or hundreds of other devices, a device could need to synchronize a massive number of updates made since the last time the device synced. In a mobile device world, the devices have varying degrees of processing power with limited amounts of memory, so the synchronization data needs to be able to be processed as a stream of data versus processing all of the data at once.

With the above desired goals, a few other properties become desirable:

  • The processing of the synchronization data should be idempotent. A device could receive the same synchronization data multiple times. Either the device needs to be able to identify that the updates have already been processed or processing the data should be idempotent. Using specialized data structures such as conflict-free replicated data types may help simplify processing.

  • Any device should be able to track what changes have been made since a local point in relative time. By keeping track of changes, a device can identify what changes were made locally which need to be transmitted to another device. Furthermore, a device could synchronize with multiple devices (or services) which requires multiple points in relative time to be able to be tracked. Note that relative time may not be a traditional clock time but could be as simple as a counter or a change token.

  • As a corollary, a local device should be able to inform another service or device what data has already been synchronized. Whether it is using (change) tokens or other means, a local device can inform another device what it has already processed to reduce the number of requests and data required to get the local device up to date.

  • Any device (including services) which have the data should have ACID transactions. ACID-compliance make synchronization easier by ensuring updates are actually processed when an operation says it is processed.

There are many other considerations for synchronization, but hopefully these thoughts give an idea on what are some of the possible complexities and desired properties when synchronizing data.