Remotes ------- Until now, all our operations were tied only to our local computer. But ``brig`` is a synchronization tool and that would be hardly very useful without supporting other peers. We call other peers »remotes« similar to the term used in the ``git`` world. A remote consists of three things: - **A human readable name:** This name can be choose by the user and can take pretty much any form, but we recommend to sticking for a form that resembles an extended email [#]_ like »ali@woods.org/desktop«. This name is **not** guaranteed to be unique! In theory everyone could take it and it is therefore only used for display purposes. There is no central place where users are registered. - **A unique fingerprint:** This serves both as address for a certain repository and as certificate of identity. It is long and hard to remember, which is the reason why ``brig`` offers to loosely link a human readable to it. - **A bunch of settings and state:** ``brig`` knows about every remote if it is online and/or authenticated. Additionally you can set a few remote-specific configuration settings like automatic updating. .. [#] To be more exact, it resembles an `XMPP or Jabber-ID `_. If we want to find out what *our own* name and fingerprint is, we can use the ``brig whoami`` command to ask a very existential questions: .. code-block:: bash # NOTE: The hash will look different for you: $ brig whoami ali@woods.org/desktop QmTTJbkfG267gidFKfDTV4j1c843z4tkUG93Hw8r6kZ17a:W1nayTG5UMcVxy9mFFNjuZDUb7uVTnmwFYiJ4Ajr1TP3bg .. note:: The fingerprint consists of two hashes divided by a colon (:). The first part is the identity of your ``IPFS`` node, the second part is the fingerprint of a keypair that was generated by ``brig`` during init and will be used to authenticate other peers. When we want to synchronize with another repository, we need to exchange fingerprints and each other as remote. There are three typical scenarios here: 1. Both repositories are controlled by you. In this case you can simple execute ``brig whoami`` on both repositories and add them with ``brig remote add`` as described in the following. 2. You want to sync with somebody you know. In this case you should both execute ``brig whoami`` and send its output over a trusted side channel. Personally, I use a `secure messenger like Signal `_, but you can also use any channel you like, including encrypted mail or meeting up with the person in question. 3. You don't know each other: Get to know each other and the proceed like in the second point. There is no way to know if somebody is the person he is pretending to be, so validate that over a separate channel - that's sadly something where ``brig`` can't help you yet. If you need to get a hint of what users use a certain domain, you can use ``brig net locate`` to get a list of those: .. code-block:: bash # This command might take some time to yield results: $ brig net locate -m domain woods.org NAME TYPE FINGERPRINT ali@woods.org domain QmTTJbk[...]:W1UDvKzjRPb4rbbk[...] Please note again: Do not blindly add the fingerprint you see here. Always make sure the person you're syncing with is the one you think they are. .. todo:: This seems currently broken as it does not yield any results. Once you have exchanged the fingerprints, you add each other as **remotes**. Let's call the other side *bob*: [#]_ .. code-block:: bash $ brig remote add bob \ QmUDSXt27LbCCG7NfNXfnwUkqwCig8RzV1wzB9ekdXaag7: W1e3rNGGCuuQnzyoiBKLdoN41yQ4NfNy9nRD3MwXk6h8Vy .. [#] The name you choose as remote can be anything you like and does not need to match the name the other person chose for themselves. It's not a bad idea though. *Bob* has do the same on his side. Otherwise the connection won't be established, because the other side won't be authenticated. By adding somebody as remote we **authenticate** them: .. code-block:: bash $ brig remote add ali \ QmTTJbkfG267gidFKfDTV4j1c843z4tkUG93Hw8r6kZ17a: W1nayTG5UMcVxy9mFFNjuZDUb7uVTnmwFYiJ4Ajr1TP3bg Thanks to the fingerprint, ``brig`` now knows how to reach the other repository over the network. This is done in the background via IPFS and might take a few moments until a valid route to the host was found. The remote list can tell us if a remote is online: .. code-block:: bash $ brig remote list NAME FINGERPRINT ROUNDTRIP ONLINE AUTHENTICATED LASTSEEN AUTO-UPDATE bob QmUDSXt27 0s ✔ ✔ Apr 16 17:31:01 no $ brig remote ping bob ping to bob: ✔ (0.00250s) Nice. Now we know that bob is online (✔) and also that he authenticated us (✔). Otherwise ``brig remote ping bob`` would have failed. .. note:: About open ports: While ``ipfs`` tries to do it's best to avoid having the user to open ports in his firewall/router. This mechanism might not be perfect though and maybe never is. If any of the following network operations might not work it might be necessary to open the port 4001 and/or enable UPnP. For security reasons we recommend to only open the required ports explicitly and not to use UPnP unless necessary. Syncing ------- Now that we added a remote, a whole new set of features are available to us. Before we move on to do our first synchronization, let's do a quick recap of what we have done so far: - Create a repository (``brig init ``) - This needs to be done only once. - Create optional mount points (``brig fstab add ``) - This needs to be done only once. - Find & add remotes (``brig remote add``) - This needs to be done once for each peer. - Add some files (``brig stage ``) - Do as often as you like. As you can see, there is a bit of initial setup work, but the actual syncing is pretty effortless now. Before we attempt to sync with anybody, it's always a good idea to see what changes they have. We can check this with ``brig diff ``: .. code-block:: bash # The "--missing" switch also tells us what files the remote does not possess: $ brig diff bob --missing • ├── _ hello.world ├── + videos/ └── README.md ⇄ README.md This output resembles the one we saw from ``brig tree`` earlier. Each node in this tree tells us about something that would happen when we merge. The prefix of each file and the color in the terminal indicate what would happen with this file. Refer to the table below to see what prefix relates to what action: ====== ==================================================================== Symbol Description ====== ==================================================================== ``+`` This file is only present on the remote side. ``-`` This file was removed on the remote side. ``→`` This file was moved to a new location. ``*`` This file was ignored because we chose to, due to our settings. ``⇄`` Both sides have changes, but they are compatible and can be merged. ``⚡`` Both sides have changes, but they are incompatible and result in conflict files. ``_`` This file is missing on the remote side (needs to be enabled with ``--missing``) ====== ==================================================================== .. note:: Remember that ``brig`` does not do any actual diffs between files, i.e. it will not show you what line changed. It does not care a lot about the content. It only records how the file metadata changes and what content hash the file has at a certain point. If you prefer a more traditional view, similar to ``git``, you can use ``--list`` on ``brig diff``. So in the above output we can tell that *Bob* added the directory ``/videos``, but does not possess the ``/hello.world`` file. He also apparently modified ``README.md``, but since we did not, it's safe for us to take over his changes. If we sync now we will get this directory from him: .. code-block:: bash $ brig sync bob $ brig ls SIZE MODTIME OWNER PATH PIN 443 B Dec 27 14:44:44 ali /README.md 🖈 443 B Dec 27 14:44:44 bob /README.md.conflict.0 12 B Dec 27 15:14:16 ali /hello.world 🖈 32 GB Dec 27 15:14:16 bob /videos You might notice that the ``sync`` step took only around one second, even though ``/videos`` is 32 GB in size. This is because ``sync`` **does not transfer actual data**. It only transferred the metadata, while the actual data will only be loaded when required. This might sound a little inconvenient at first. When I want to watch the video, I'd prefer to have it cached locally before viewing it to avoid stuttering playback. If you plan to use the files immediately, you should be using pinning (see :ref:`pinning-section`) Data retrieval ~~~~~~~~~~~~~~ If the data is not on your local machine, where is it then? Thanks to IPFS it can be transferred from any other peer that caches this particular content. Content is usually cached when the peer either really stores this file or if this peer recently used this content. In the latter case it will still be available in its cache. This property is particularly useful when having a small device for viewing data (e.g. a smartphone, granted ``brig`` would run there) and a big machine that acts as storage server (e.g. a desktop). How are the files secure then if they essentially could be everywhere? Every file is encrypted by ``brig`` before giving it to IPFS. The encryption key is part of the metadata and is only available to the peers that you chose to synchronize with. Think of each brig repository only as a cache for the whole network it is in. Partial synchronisation ~~~~~~~~~~~~~~~~~~~~~~~ Sometimes you only want to share certain things with certain people. You probably want to share all your ``/photos`` directory with your significant other, but not with your fellow students. On the other hand you maybe want to share the ``/lectures`` folder with them. In ``brig`` you can define what folder you want to share with what remote. If you do not limit this, **all folders will be open to a remote by default.** Also note, that if a remote already got some content of a folder you did not want to share, he will still be able to access it. If you're unsure, you should better be restrictive than too permissive. To add a folder for a specific remote, you can use the ``folders`` subcommand of ``brig remote``: .. code-block:: bash # Starting with next sync, bob will only see the /videos folder: $ brig remote folder add bob /videos $ brig remote folder ls bob /videos If you're tired of typing all of this, be reminded that there are very short aliases for most subcommands: .. code-block:: bash $ brig rmt f a bob /videos In some cases you might not trust your peers with some folders or don't want to have modifications in that specific folder. For this case, ``brig`` supports adding a folder as ``--read-only``. Other remotes still will have access to the folder, but whenever we sync with them the changes they made are ignored. You can add a read-only folder by adding the ``--read-only`` switch to the command above: .. code-block:: bash $ brig rmt f a bob /videos --read-only .. note:: If you want to overwrite an existing folder with new settings, you can use the ``set`` subcommand: .. code-block:: bash $ brig remote folder set bob /videos -c embrace --read-only See below for explanation on those additional options. Conflicts ~~~~~~~~~ Whenever two repositories have a file at the same path, ``brig`` needs to do some conflict resolving. If those files are equal or if they share common history and did not diverge there is nothing to fear. But what if both sides have different versions of a file without common history? In this case ``brig`` offers you to handle conflict by one of the three strategies: * ``ignore``: Ignore the change from the remote side. * ``embrace``: Ignore our state and take over the remote's change. * ``marker``: Create a conflict file with the same name but a ``.conflict`` ending. Leave it to the user to resolve the conflict. This is the **default.** You can configure this behavior by using ``brig cfg``: .. code-block:: bash $ brig cfg set fs.sync.conflict_strategy marker In some cases this might not be enough though. Sometimes you might want to say »I trust this remote, always accept their changes«. You can do this by setting the conflict strategy per remote. If no specific conflict strategy is set, ``fs.sync.conflict_strategy`` is used. You can set the strategy by using a subcommand of the ``brig remote`` family: .. code-block:: bash # Always take the versions of bob on conflicts: $ brig remote conflict-strategy embrace bob Still not enough? You can also set the conflict strategy per folder. This will trump the per-remote folder strategy: .. code-block:: bash # Use the default in all folders but use "embrace" in this one: $ brig remote folder add bob /collab -c embrace Automatic Updating ~~~~~~~~~~~~~~~~~~ .. warning:: This feature is experimental and builds upon the also experimental pubsub experiment of the IPFS project. Use with care. If you do not want to hit ``brig sync`` every time somebody in the network changed something, you can enable the automatic updating for any remote you like. Let's suppose we are ``ali`` and want to receive updates on every change of ``bob``, we should simply add the following: .. code-block:: bash $ brig remote auto-update enable bob # (You can also abbreviate most of that:) # brig rmt au e bob Alternatively, we could have used the ``-a`` switch when adding ``bob`` as remote: .. code-block:: bash $ brig remote add bob -a In any case, an initial sync is performed with this remote and a sync on every change that ``bob`` published. Keep in mind that ``bob`` will not receive your updates by default, he needs to decide to use auto updating for himself. You can watch the times when your repository was updated automatically by looking at ``brig log``: .. code-block:: bash $ brig log - Sun Dec 16 18:24:27 CET 2018 • (curr) W1kGKKviWCBY Sun Dec 16 18:24:27 CET 2018 sync due to notification from »bob« (head) ... Pushing changes ~~~~~~~~~~~~~~~ As you saw above, doing a ``brig sync`` won't do a bidirectional synchronisation. It will only fetch metadata from the remote and modify our local state with it. In some cases you might want to push data to a remote - especially when it is on one of your machines and you use for example as archival repository. By default pushing to a remote is rejected. You can enable it on a per-remote basis with this command out of the ``brig remote`` family of commands: .. code-block:: bash # Allow bob and charlie to auto push to us. $ brig remote auto-push enable bob charlie Now either ``bob`` or ``charlie`` can do this from their machines: .. code-block:: bash # bob's machine: $ brig push ali This will simply ask ``ali`` to do a sync with ``bob``.