PerServ - Hydra Sync app


Overview


Conceptual diagram. Reduced sized images may be opened full size by clicking image.
Hydra is an app for doing 1-way sync (also known as "mirroring") across the internet. Common uses for this type of sync are doing backups or distributing code.

In hydra terminology, the computer sourcing the instructions for file transfer is known as a "capitol" and the computer retrieving and acting on the instructions is known as a "city". The file transfer flow always into the city as a destination. City files are updated only by actions initiated by the city. The source of the files may or may not be the capitol.

A source computer must have a visible, static IP address with SSL certificate that the destination city can access so that the transfer is secure. Insecure http transfer is also supported but is best reserved for dynamic private addresses on a LAN.

In the diagram, computers 1 and 4 have public ip addresses and SSL certificates. Thus, they naturally serve as capitols. If we stipulate that the connection at the top of 1 flow coming from the top, then both 1 and 4 serve a dual role: city and capitol.

Synch flows outward from 1 and 4. Computers 2, 3, 6, and 8 can all make https requests to 1 and 4. The flow is always "pull" rather than push. In this way, PerServ accommodates computers which may be behind NAT gateways, may have no public address, or not have an SSL certificate.

The highway from 3 to 5 to 7 of necessity takes place over http because there is no public ip address with an SSL certificate in the chain.

Below, we discuss the steps in setting up a highway between capitol and city.

Installing SSL (Capitol)


Edit the pem_file_name and my_domain entries in control.xml
SSL needs to be installed for a Capitol which is accessed via https. You may skip this step the pem_file_name if your computer is an http-only capitol or a city.

The PEM format certificate itself is placed in the perserv/iam directory.

The name of the PEM file is entered in

Admin >> XML Edit >> Control.xml >> server >> pem_file_name

The ">>" notation separates clicks or selections. So, "Admin >> XML Edit" is shorthand for "Under, the Admin menu, click the XML Edit link".

Enter the domain name (CN or common name in the certificate) in

Admin >> XML Edit >> Control.xml >> server >> my_domain

If you do not have a domain name, use your permanent ip address. If you do not have a permanent ip address, use 127.0.0.1.

Setting up Hydra port pair (Both)


There must be a port pair with hydra enabled to use hydra.
Fixed port pair 49996 is reserved for Hydra use so that all computers on a network have the same port pair available. Alternatively, you can assign a new pair to hydra. The hydra user interface and machine-machine communication use the same port pair. It is advised you avoid double use of the owner port and http settings in particular.

The screen shot at right is taken after the pair protocol/function pull-down is changed to http-https and the port_pair number clicked. Give the port a name and enable http and/or https. During development, it is helpful to use http to remove ssl type problems until the basic communication is established. It is also helpful to enable the admin menu to use the port manager on the new port pair (which is what we are looking at here albeit on the owner port). After hydra is working, remove admin access to the the hydra port pair.

From the control.xml my_domain setting and the port manager settings here, the http address of the port being set up is:

http://luna.telemsgpad.com:45000/33570-68100/

and the https address is:

https://luna.telemsgpad.com:45001/44412-38242/

We will need at one of these addresses to log in and to use when setting up cities. After changes made, similar to those shown, click OK_proceed. Reboot the server if directed to do so onscreen and close the owner browser window.

Log in to Capitol port (Capitol)


Welcome message after login
Open a browser window using one of the addresses above. You should see a welcome screen similar to above. You must see a Hydra menu to continue on. Whether you see other menus is a function of how you set up the port pair just above.

Hover over the Hydra menu and select hydra_capitol_edit.

Capitol settings (Capitol)


Example setup of capitol.
Here we show a setup for two cities: city1 and city2. You may choose any meaningful names not containing special characters (may not include spaces either) instead of city1 or city2.

There are 4 tags for each city. There are two settings specific to the source, one specific to the destination, and one shared.

The src_acs parameter identifies an http or https address of a hydra port pair. See the ACS backgrounder pagefor more information about the acs. The acs ends with a "/" and does not contain any fields past the password. The src_acs often specifies the machine hosting the capitol file but it could refer to any hydra port where the port and password are known.

The src_top_dir contains the top-most directory to be transferred.

The dest_top_dir is parallel to the src_top_dir and will end up mirroring selected directories. We select which subdirectories to mirror by naming them in the sub_dirs list. For example, ["a", "c", "e"] specifies the a, c, and e subdirectories under both src_top_dir and dest_top_dir. We can specify [] to mean all subdirectories.

It is useful debugging to initially specify a subdirectory such as ["test"] to insure everything is set up correctly. Say you intend to transfer everything from boo/hoo to bat/man. If you make a mistake during development, you could make wide scale unintended changes. Instead, specify sub_dirs as ["test"]. This limits transfers and deletions between only boo/hoo/test and bat/man/test. Once this works as intended, you can change sub_dirs from ["test"] to []. Now everything in src_top_dir is transferred to dest_top_dir.

City settings (City)


Example setup of city
Here we show the setup of a city. There are two parameters to fill in. Neither of them is arbitrary. Both have been defined by our capitol setup.

The cap_acs is on of the links derived during the Setting up Hydra port pair. The ia_dest is one of the tags lying between highway tags in the Capitol settings. There we defined city1 and city2 and here we use city1.

Note that the structure of capitol, source, and destination allows for specifying 3 separate computers. For better centralized management, you would put all the capitol specs on a single computer. For better robustness in case of central failure, you could also distribute the capitol specs to reside on same computer as city.

Troubleshooting

When setting up a multi-computer network it is easy to make mistakes and difficult to isolate the problem(s). We have a Net Management page describing the tool provided to troubleshoot and visualize operation of your network.


Closing thoughts

The design here reflects years of experience and frustration with real synchronization tasks across the internet. Here "real" means corruption, interruptions, outages, etc. One way mirroring is robust in a way that 2-way synchronization just isn't. Hydra supports one way mirroring because it works reliably.

If you need 2-way synchronization, we urge you to adapt your use case to 1-way synchronization. For example, if machines A, B, and C can all write to the same file, split the file into 3 parts, a, b, and c with each machine writing its own private file. You can now 1-way synch so that every machine has every file. Now, let's say C goes down. Machines, A and B have an old copy but they are not writing to c. When C comes back up, it updates the copy of c.

The diagram here shows a directed acyclic graph (no paths close in on themselves). Hydra is intended to implement such a graph. However, even in a very complicated network with inadvertent closed loops and conflicting updates chasing each other around a closed path, most sync schemes should eventually settle out. It may take breaks in internet connections in the worst case to insure settling into a consistent state but breaks are a given in real networks.