a little class for combining behavior, layout and style

2019-04-30

I like some of what React offers. I also think it's bloated, over-hyped, over-engineered and it sees the web as a compile target rather than a development platform.

I like most of what Web Components offer, they're a part of the web platform. They offer true component encapsulation  —  without hacks. They are the result of consensus among the web's working groups. And like most APIs designed by consensus, they are a little awkward.

img

Tonic is about 250 lines of code. It borrows goals and ideas from React but is built on native Web Components. It works in all browsers. It's the minimum of what is needed to organize application code, the flow of data and accommodate component based architecture.

You can find the core library here and a collection of components here on Github.

What are javascript then-ables

async / await improves program flow and reduces the number of callbacks in your code. The await keyword can be used to pause the current code path and wait for an async function to return a value without blocking the event loop.

async function main () {
  console.log('started')
  await sleep(100)
  console.log('finished')
}

main()

An implementation for the sleep function might look like this...

const sleep = n => new Promise(r => setTimeout(r, n))

However, as this example demonstrates, the code in the promise starts executing immediately. Promises are eager (as opposed to being lazy), and sometimes we want them to be lazy. Here is a detailed explaination of why an eager promise may not be what you want.

A then-able is lazy. It has no constructor. It's any function, object or class that implements a then method.

Await-able Classes

To create an async class, implement a then method on it!

class Foo {
  then (resolve, reject) {
    resolve(42)
  }
}

async function main () {
  const answer = await new Foo()
  // answer === 42
}
main()

Await-able Objects

You can do the same thing with an object. You can name the callback functions whatever you want. Also, you aren't required to use or care about the rejection callback.

const Foo = {
  then (resolve) {
    setTimeout(() => resolve(42), 1024)
  }
}

async function main () {
  const answer = await Foo
  // answer === 42
}
main()

Await-able object factories

const Foo = num => ({
  then (resolve) {
    setTimeout(() => resolve(num), 1024)
  }
})

async function main () {
  const answer = await Foo(42)
}
main()

Async then-ables

Object and class methods can use the async keyword, just like functions.

const Foo = {
  async then (resolve) {
    resolve(await request('https://foo.com'))
  }
}

Destructuring assignments provide a way to return multiple values...

class Foo {
  then (resolve) {
    request('https://foo.com', (err, res) => resolve({ err, res }))
  }
}

async function main () {
  const { err, res } = await new Foo

  // More than one err? Const is block-scoped!
  {
    const { err, res } = await new Foo
  }

  // Destructured values can also be aliased.
  const { err: namedError, res: namedResponse } = await new Foo
}
main()

preamble

synopsis

This is a collection of notes that explore peer-to-peer topics.

description

Rhis collection focuses on the following topics...

These notes are not complete and don't advocate any particular approachs. They are related to my work on dat-cxx.

illustrated lamport timestamp

problem

With the client-server model, you can easily determine the order of events in a system because they are all maintained by a single source. This is critical in, for example a chat application.

But with the distributed model, how do we know if an event happened before another? How can we thread together datasets from different sources in the correct order?

solution

A Lamport Timestamp is one solution to determine the order of events in a distributed system. Although it may not solve all problems in this problem space, it is a useful primitive that we will explore.

Clocks vs. Logical Clocks

Why don't we use regular time stamps? Most clocks count time at different rates and experience failures that require resynchronization. This means they are reliably unreliable for determining the order of events.

Lamport Timestamps use a Logical Clock to keep track of the order of events on each node. A logical clock is not a clock that keeps track of the time, it's a monotonically increasing counter. So, when a node in a network receives a message, it re-synchronizes its counter (its clock) with the node that sent the message.

Example

Node A increments its clock before each event that hapens. An event is something meaningful, like when it creates some data. When node A eventually sends its payload over the network, it will include the current value of its clock.

let clock = 0

//
// A thread-safe, monotonically increasing function.
//
function createTimeStamp () {
  clock += 1
  return clock 
}

function doSomething (data) {
  //
  // Do something with some data.
  //
  return {
    data,
    clock: createTimeStamp()
  }
}

//
// Eventually send the data to the network.
//
sendToNetworkQueue(doSomething({ ... }))

When node B receives a message from node A, it will decide how to set its own clock. If the clock value in the message is greater than its own value, it will use the value in the message. Otherwise it will use its own value. In either case, it will also increment its own clock by 1.

let clock = 0

//
// Eventually receive some data from the network.
//
receiveFromNetworkQueue (message) {
  clock = Math.max(message.clock, clock) + 1
}

Here we semi-randomly fail to always tell the other node about the event that happened, illustrating what happens when a node is eventually synchronized.

This may not be the correct primitive for all your use cases. For example, Lamport Timestamps don't express causality, meaning, the reason why one event happened before another isn't in scope of this soluton, but that is something that can be achieved using a Vector Clock.


Special thanks to Fedor Indutny and Feross Aboukhadijeh for reviewing this post. ♥

vector clocks

In the previous post, I wrote about how Lamport Timestamps (aka Logical Clocks) can help determine the order of events in a distributed system.

problem

Logical clocks only offer "Partial Ordering", because they can tell us the order of a single event, but not the total ordering of events or why a system arrived at its state.

solutions

Vector Clocks build on the idea of Logical Clocks to help track causality in a distributed system.

Here is an example vector clock in a network where there are three participating nodes...

{ alice: 0, bob: 1, carol: 0 }

To set up a node we will give it an id and an in memory object to store some data.

const myId = 'alice'
const data = {}

sending messages

When a node writes some data, it increments its own logical clock in the vector and includes it as a property of a message that it will attempt to send. We also add the value as a property of the message.

function write (key, value) {
  if (!data[key]) {
    data[key] = { clock: { [myId]: 0 } }
  }

  data[key].clock[myId] += 1 
  data[key].value = [value]

  send(key, data[key])
}

In this case we made the value property an array. This is because we must anticipate the possibility of concurrent messages — that is, a message was received where two nodes have a logical clock with the same count.

In this case we can push the new value onto the array and allow the the conflict to be resolved somehow (we'll discuss this more later).

receiving messages

When a node receives a message it increments its own Logical Clock in its local copy of the vector.

Then for each node in the message's vector, it compares the local clock count (if there is one) to the clock count in the received message, taking the max of the numbers.

const max = arr => Math.max.apply(null, Object.values(arr))

function receive (message) {
  const key = message.key

  //
  // If this is new data, there is no need to compare anything.
  // we can store it locally and return early from the function.
  //
  if (!data[key]) {
    data[key] = message
    data.clock[myId] = max(message.clock) + 1
    return
  }

  //
  // We have received the message, update our clock
  //
  data[key].clock[myId] += 1

  const localClock = data[key].clock
  const messageClock = message.clock

  //
  // For each node in the vector of the message
  //
  for (const id in Object.keys(messageClock)) {
    const a = localClock[id] || 0
    const b = messageClock[id]

    const isConcurrent = a === b

    if (isConcurrent) {
      data[key].conflict = true
      data[key].value.push(message.value)
      continue
    }

    const happenedBefore = a < b

    if (happenedBefore) {
      data[key].value = [message.value]
    }

    localClock[id] = Math.max(a, b)
  }
}

handling concurrent messages

Two messages that are received at the same time and have the same logical clock count are "concurrent".

To understand what to do with this type of data, we need to create a resolution function. This function may be the only way to determine what data is either a descendant or which data comes-before.

  1. Reject the data and send it back to the clients asking for it to be resolved. This might mean asking them to manually merge or discard some of the data.

  2. Last-Writer-Wins uses time-based timestamps. If you consider clock-drift (mentioned in the first post), there is a high probability of losing data with this strategy.

research timeline

When discussing Vector Clocks we should consider some other closely related research...

Lamport Timestamp 1978 Version Vector 1983 Vector Clock 1988 Bound Version Vector 2004 Interval Tree Clock 2008 Dotted Version Vectors 2010

Version Vectors also build on the idea of Lamport Timestamps, but are specifically meant to track changes to data in distributed systems. They are also the basis for optimistic replication.

disadvantages

Each message sent by a node contains a vector clock that has all the node names (and their corresponding clock counts) who want to write to the same field of data.

This can be a problem since a data structure that can grow to an unbound size can be a problem in larger networks with more frequent writes. Strategies for dealing with this are often based on what suits your use-cases best, for example, two possible solutions are...

  1. If a network has a finite number of nodes, a message that has reached all nodes can be considered "complete", could be marked as such and have it's historic information removed.

  2. If a network has an acceptable threshold of nodes that once a message has reached, the message can be considered complete and can then be cleaned up.

implementing dat

In my spare time I am implementing dat by following this, this and this as references.

You can follow this post and this github org for updates and information.

DTN recap

cafe dog

History

Over the last 10 years I've been a part of various meet-ups, IRC channels and p2p networks that are interested in building distributed systems. In 2015 I pulled together a lot of the leaders and contributors from the projects that I found interesting for an event we called DTN, aka Data Terra Nemo, Decentralize The Network, Don't Think Normal. It went well!

2019

About 6 months ago Feross, Mikeal and Juan convinced me I should do another one and it made sense! After all, many of the projects we discussed in 2015 were just ideas (libp2p), still in the prototyping phase (dat) or didn't exist at all (filecoin, patchwork). It happened!

This event works for a few reasons.

  1. Zero profit. 100% of what we earn gets spent on the conference — no exceptions. All funds go towards flights, hotels, food, A/V, etc.

  2. We curate speakers who are hard working, highly motivated implementers who are also kind and empathetic people. Software is nothing without the people who work together to make it.

  3. One of the most important reasons this works is that it's collaborative and not competitive. We're sharing fundamental ideas. And while we're taking different approaches to solving some of the same problems, we're all interested in the best outcome. Things like programming languages are trivial details.


What happened?

It was a 2 day event. About 150 people attended. We recorded about 12 hours of video from 12 speakers. We had 12 formal talks and several impromptu lightning talks. I'll be posting the videos online once they are processed.

We discussed technical (and some of the social) challenges in coding fundamental primitives that help people build a more distributed and decentralized internet. We shared language agnostic patterns for building solutions but also many concrete solutions — actual code that illustrates what problems we're solving today.

Personally I walked away with new ideas, feeling more educated and connected to the people from my community. I heard the same from a lot of people who attended.

How did you make this happen?

I have no idea. It wasn't easy. Unexpected things happened. People are hard to organize. I'm not an event organizer. I have no idea what I'm doing. Thankfully I had an amazing co-organizer, Aprile. I almost broke-even too. But ultimately it was a lot of fun. The lesson here is that anyone can do this if they want to. If you want help putting on your own event, I'm happy to discuss what I know! My handle is heapwolf most places on the internet. If you're not on any decentralized networks yet, you can even reach me on twitter.

What's next?

Let's do it again! Many of the projects we met to discuss are moving faster, more people are involved, so let's not make it a 4 year gap this time. Should we do the same dates? Different Location? Let's talk.

Thank you!

If you bought a ticket, you were a sponsor! You made this event happen. So I want to thank all the sponsors, especially those who bought high value tickets. You are helping to build a better internet for everyone.

CODE

GITHUB

https://github.com/heapwolf

PUBLIC KEY

Curriculum Vitae

Personal

Name: Paolo Fragomeni, Software Engineer

Contact

Email: paolo@async.ly

Web: https://hx.ht

Twitter: https://twitter.com/heapwolf

Github: https://github.com/heapwolf

Summary

I Left MIT in 2010 to co-found Nodejitsu (a PaaS, since integrated with GoDaddy). Most recently I founded Voltra Co. (entertainment software) which joined Conductor Lab. In addition to being a technical founder, CTO and engineer I have worked in the infosec space.

Expertise

Computer Science Research. Software Engineering: programming design and engineering, concurrent and distributed programming, metaprogramming, functional programming and ECMAScript (Javascript). Key-Value store

Experience

CTO, Cofounder at Voltra Co.

January 2016 - Augest 2018 (2.5 years)

Voltra Co. was a cloud storage service and set of audio products. Voltra's desktop and mobile players sync so you can stream your music from anywhere. The only ad-free fully hi-res platform. On this project I worked with Electron, JavaScript, Node.js, C++, CSS3, HTML5, Stylus, Jade, Webpack, React, React-Native, and Amazon Web Services. Voltra joined Conductor Lab in Augest 2018.

VP of Engineering at Now Secure

November 2014 - January 2016 (1 year 3 months)

Built engineering and security research teams. Coordinated engineering and research teams. Set technical goals, worked hands on on lots of projects. On this project I worked primarily with C, C++, JavaScript, Node.js, HTML5.

Engineer, CTO at Mic

January 2014 - November 2014 (11 months)

Hereishow.to joined mic.com where I served as CTO. Built an engineering team and integrated components of Here Is How into existing products. On this project I worked with Amazon Web Services, Node.js JavaScript, HTML5, CSS3.

Engineer, CTO, Cofounder at Here Is How

November 2012 - January 2014 (1 year 3 months)

A CMS for technical writing, a web based interface similar to Medium.com. This project was acqui-hired by mic.com On this project I worked with Docker, JavaScript, Node.js, Websockets, C, C++, HTML5, CSS3.

Engineer, CTO, Cofounder at Nodejitsu

September 2010 - December 2012 (2 years 4 months)

Co-Founder, Chief Technology Officer. Lots of R&D. Conceptualized and implemented products that simplify and manage application deployments for the node.js platform.