dreamful-service

Let's dream together!

Getting Started

  1. Have a recent version of OpenJDK (11+).
  2. Install leiningen.

Local Usage

lein run

Storage Options

If you shut your server down without configuring storage, everything you've stored will disappear. If you want data to persist between startups, you will need to turn on persistence.

SQLite Persistence

The baseline persistence is sqlite. Here's how to persist to a file on disk:

PERSISTENCE_TYPE=sqlite-disk
STORAGE_LOC=./dreamful.db

This will create a sqlite database called dreamful.db in the current working directory.

Backups

If you don't want to manage backups yourself, and potentially upload files that are in the midst of being written to, you can instead tell the server to write journal backups to S3:

S3_BACKUP_BUCKET=my-backup-bucket-name

Layering on IPFS

You can cause the core content of your profiles and dreams to be stored in IPFS like this:

RECORD_PERSISTENCE_TYPE=ipfs
IPFS_URL=http://127.0.0.1:5001/api/v0/

Then, we will use baseline persistence to store metadata like the latest hash for each stream. Your clients can then interact with the server like normal, or they can choose to interact with IPFS directly by setting a header on every request: x-ipfs-load-disabled: true -- this will cause invocations to GET endpoints to retrieve the necessary metadata about where to get content in IPFS instead of the content itself.

Currently, mobile clients only know how to fetch data from IPFS. Therefore, any server that wants to be accessed by a mobile client should use this model.

Eventually, the service will be federated. This would mean that users could host their own service which their mobile client will connect to. That service would run a local IPFS node that users of that service would use to fetch data for dreams and profiles across the entire federation.

The swarm key is available at https://dreamful.app/ipfs/swarm.key

> shasum swarm.key
0a8b426b5e0ab018bb1324509fca67d4987b7e3b  swarm.key

Using an IPFS Gateway

You can point Dreamful at an IPFS Gateway. Just set:

IPFS_GATEWAY_URL=...

And you should be all set. Note that, for storage, you should still set an IPFS_URL

Another thing to remember is that, if you are using a local IPFS Gateway for development, you are likely to run into issues with the gateway URL. This is because by default the gateway does domain redirects that don't work with Java.

In order to disable redirects you'll need to update the IPFS config with the following structure:

"Gateway": {
    //...
	"PublicGateways": {
		"localhost": {
			"NoDNSLink": false,
			"Paths": [
				"/ipfs",
				"/ipns"
			],
			"UseSubdomains": false
		}
	}
}

Externally Facing (EXPERIMENTAL)

Sometimes you want to run a server and let other folks connect to it. You can do this by turning on UPnP support. We will attempt to open a port to the outside internet automatically. Here's how:

UPNP_ENABLED=true lein run

Tests

This service has been developed using a literate testing approach using itl. The result of executing this file can be seen at the Test Results page.

You can run the tests using lein itl

Here are its tests:

FileResultPassFailException
server.mdoutput600
devices.mdoutput540
profiles.mdoutput1000
dreams.mdoutput2800
dream-messaging.mdoutput2200
(execute-pages: indir=test/itl, outdir=docs, logfile=itl.log)

For lower-level primitive tests (such as baseline cryptography), we have a series of unit tests using clojure.test. These can be found in test/clj. You can run those tests using lein test

To run all the tests, simply run bin/test-all. The unit tests and itl tests will run in parallel along with all linting and artifact maintenance jobs.

Preparing to Release

If you're ready to release the software, run bin/prepare-release. This is how we ensure the software is always in a releasable state, and should be run before pushing to origin/master.

This will create an artifact you can run:

java -jar target/uberjar/dreamful-service.jar

License

Copyright © 2021 Stephen Starkey

All Rights Reserved

Cryptography Notice

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

Credits

ASCII Art from http://www.chris.com/ascii/index.php?art=nature/astronomy