As people quickly discover, implementing RELOAD is not an easy task – the specification is complex, covers multiple network layers, is extensible and flexible and the fact that security is mandatory creates even more challenges at debug time. This is why new implementations generally focus on having the minimum set of features working between nodes using the same software.
Making two different RELOAD implementations interoperate requires a lot more work, mostly because connecting to a RELOAD overlay is not as simple as providing an IP address and port to connect to. Because of the extensibility of RELOAD, all the nodes in an overlay must use the same set of parameters, parameters that are collected and distributed in an XML document that need to be cryptographically signed. In addition to this, all nodes must communicate over (D)TLS links, using both client and server certificates signed by a CA that is local to the overlay. Configuration file and certificates must be distributed to each node and when two or more implementations wants to participate in the same overlay, adhoc methods to provision these elements are no longer adequate. The standard way to do that is through a configuration and enrollment server but unfortunately that is probably the part of the RELOAD specification that most implementers would assign the lowest priority, thus creating an higher barrier to interoperability testing than one would expect.
This is why during the last RELOAD interoperability testing event in Paris, I volunteered to provide configuration and enrollment servers as a service to RELOAD implementers, so they do not have to worry about this part. I already had my own configuration and enrollment servers, but I had to rewrite them from scratch because of two additional requirements: They had to work with any set of parameters, even some that my own implementation of RELOAD do not support yet, and it must be possible to host servers for multiple overlays on the same physical server (virtual server). A first set of servers are now deployed and in use by the participants of the last RELOAD interoperability event, so it is now time to open it to a larger set of participants.
First what this service is not: It is not to host commercial services, and it is not meant to showcase implementations. The service is free for RELOAD implementers (up to 5 overlays per implementer) for the explicit purpose of letting other implementers connect to your RELOAD implementation, which means that you are supposed to provision a username/password for any other implementer on request, on a reciprocity basis. Contact me directly if you are interested in an usage that does not fit this description.
The enrollment for the service is simple: send me an email containing the X500 names that will be used to provision your servers. Here’s an example to provision a fictional overlay named “my-overlay-reload.implementers.org”:
C=US, ST=California, L=Saratoga, O=Impedance Mismatch, LLC, OU=R&D,
CN=my-overlay-reload.implementers.org
The C=, ST=, L=, O= and OU= components should describe your organization (not you). The CN= component contains the name requested for your overlay. Note that the “-reload.implementers.org” part is mandatory, but you can choose to use whatever name before this suffix, as long as it is not already taken, that it follows the DNS label rules and that it does not contain a dot (wildcard certificates do not support sub-subdomains).
With these information I will provision the following:
- The DNS RR as described in the RELOAD draft
- A configuration server.
- An enrollment server, with its CA certificate
- A secure Operation, Administration and Management (OAM) server.
The DNS server will permit to retrieve the IP addresses and ports that can be used to connect to the configuration server. If we reuse our example above, the following command will retrieve the DNS name and port:
$ host -t SRV _reload-config._tcp.my-overlay-reload.implementers.org
_reload-config._tcp.my-overlay-reload.implementers.org has SRV record 40 0 443
my-overlay-reload.implementers.org.
Note that the example uses the new service and well-known URL name that were agreed on in the Vancouver meeting, but the current name (p2psip-enroll) will be supported until the updated specification is published.
The DNS name can then be resolved (the IPv6 address is functional):
$ host my-overlay-reload.implementers.org
my-overlay-reload.implementers.org has address 173.246.102.69
my-overlay-reload.implementers.org has IPv6 address
2604:3400:dc1:41:216:3eff:fe5b:8240
Then the configuration file can be retrieved by following the rules listed in the specification:
$ curl --resolve my-overlay-reload.implementers.org:443:173.246.102.69
https://my-overlay-reload.implementers.org/.well-known/reload-config
The returned configuration file will contain a root-cert element containing the CA certificate that was created for this overlay, and will be signed with a configuration signer that will be maintained by the configuration server. Basically the configuration server will automatically renew the configuration signer and resign the configuration file every 30 days, or sooner if you upload a new configuration file (more on this later). Note that to ensure that there is no lapse in the rollover of signer certificates, the configuration file must be retrieved periodically (the expiration attribute contains the expiration date of the signer certificate, so retrieving the configuration document one or two days before this date will guarantee that any configuration file can be used to validate the next one in sequence). This feature frees the implementers from developing its own signing tools (a future version will permit the implementers to maintain their own signer and to upload a signed configuration file).
The configuration file also contain an enrollment-server element, pointing to the enrollment server itself, that can be used to create certificates as described in the specification. The enrollment server requires a valid username/password to create a certificate and anyway the default configuration document returned is filled with the minimum parameters required, so they are useless as it to run a real overlay. Modifying the configuration document and managing the users that can request a certificate (and so join the overlay) is the responsibility of the OAM server.
Because the OAM server uses a client certificate for authentication, it uses a different domain name than the configuration and enrollment server. The domain name will use the “-oam-reload-implementers.org” suffix, and will use a separate CA to create the client certificate, so a user of the overlay cannot use its certificate to change the configuration (That would be a good idea to define a new X.509 extended key usage purpose for RELOAD to test for this).
The OAM server uses a RESTful API to manage the configuration and enrollment servers (well, as RESTful as possible, because the API is in fact auto-generated from a JMX API, and I did not find another solution that to map a JMX operation to a POST. But more on this in a future blog entry). Here’s the commands to add a new user, change a user password, list the users and remove a user:
$ curl --cert client.crt --key client.key --data "name=myname&password=mypassword"
https://my-overlay-oam-reload.implementers.org/type=Enrollment/addUser
$ curl --cert client.crt --key client.key --data "name=myname&password=mypassword"
https://my-overlay-oam-reload.implementers.org/type=Enrollment/modifyUser
$ curl --cert client.crt --key client.key https://notomele-reload.implementers.org/type=Enrollment/Users
$ curl --cert client.crt --key client.key --data "name=myname"
https://my-overlay-oam-reload.implementers.org/type=Enrollment/removeUser
The password is stored as a bcrypt hash, so it is safe as long as you do not use weak passwords.
The last step is to modify the configuration, probably to add a bootstrap element element. Currently the OAM server manages what is called a naked configuration, which is a configuration document stripped of all signatures. The current naked configuration can be retrieved with the following command:
$ curl --cert client.crt --key client.key https://my-overlay-oam-reload.implementers.org/type=Configuration/NakedConfiguration > config.relo
The file can then be freely modified with the following constraints:
- The file must be valid XML and must conform to the schema in the specification (including the use of namespaces).
- The sequence attribute value must be increased by exactly one, modulo 65535.
- The instance-name attribute must not be modified.
- The expiration attribute must not be modified.
- A node-id-length element must not be added.
- The root-cert element must not be removed or modified, and a new one must not be added.
- The enrollment-server element must not be removed or modified, and a new one must not be added.
- The configuration-signer element must not be removed or modified, and a new one must not be added.
- A shared-secret element must not be added.
- A self-signed-permitted element with a value of “true” must not be added.
- A kind-signer element must not be added.
- A kind-signature or signature element must not be added
- Additional configuration elements must not be added.
Then the file can be uploaded into the configuration server:
$ curl --cert client.crt --key client.key -T config.relo https://my-overlay-oam-reload.implementers.org/type=Configuration/NakedConfiguration
If there is a problem in the configuration, an HTTP error 4xx should be returned, hopefully with a text explaining the problem (please send me an email if you think that the text is not helpful, or if a 5xx error is returned).